diff --git a/nvx-rs/README.md b/nvx-rs/README.md index f3a9ea90d..8c85067d1 100644 --- a/nvx-rs/README.md +++ b/nvx-rs/README.md @@ -76,6 +76,16 @@ The [module diagrams](../doc/modules/README.md) show the crate's modules, the modules that each contains, and the dependencies between its top-level modules, on Linux and Windows. +The [filesystem module](src/host/filesystem/mod.rs) keeps its request types and +planner together, with private siblings for guest and host paths, access policies, +OpenVMM limits, and recorded-plan checks and rendering. The +[instance module](src/host/instance/mod.rs) separates private-file helpers, +lock-held lifecycle transitions, and workload execution. The CLI's +[managed sandbox lifecycle](src/host/cli/sandbox/lifecycle/mod.rs) separates state +directories, outcome reports, start and exec, and stop and deprovision. Their +`mod.rs` files retain the existing entry points through re-exports; their +`tests.rs` files keep the unit tests at the same module paths. + | Module | Contents | | --- | --- | | `nvx::protocol` | The codec of the control console: `NVXS` outer records, `NVXC` application frames, the `EXEC` payload that the host encodes and the guest agent decodes, the `MAPS` payloads of the host mapping table, `GuestFeatures`, and the `nvx_maps=` and `nvx_workload_account=create` tokens. The [conformance vectors](tests/vectors/README.md) of version 1, `tests/vectors/nvxc-v1.txt`, and OpenVMM's vectors and cases of the `NVXS` session pin every record, frame, and payload independently of the codec, which the host, the guest agent, and the fake OpenVMM share | diff --git a/nvx-rs/src/host/cli/sandbox/lifecycle.rs b/nvx-rs/src/host/cli/sandbox/lifecycle.rs deleted file mode 100644 index 73ad70a3d..000000000 --- a/nvx-rs/src/host/cli/sandbox/lifecycle.rs +++ /dev/null @@ -1,1096 +0,0 @@ -//! The lifecycle of a managed sandbox, a port of the former -//! `scripts/nvx_tools/sandbox_lifecycle.py`, which it replaced (microsoft/nvx#359), on the -//! [`store`](crate::host::store), [`supervisor`], and [`session`](crate::host::session) that -//! every host of a sandbox shares. -//! -//! A sandbox lives in its state directory, which is the directory of its store: -//! -//! ```text -//! / -//! config.json the launch that provision records, in format 7, with the export plan of -//! its shares or `null` without shares, the bindings of its distro layer -//! and of its scratch image, and its network -//! sandbox.lock the store's lock, which serializes the lifecycle transitions -//! launch.json the store's journal of a start in progress, or of an interrupted one -//! runtime.json the store's runtime record of the running VM, in its format 4 -//! control.capability the 32-byte capability of the running OpenVMM -//! control.sock the control endpoint of the running OpenVMM, on POSIX -//! openvmm.log OpenVMM's output, which each start replaces -//! outcome.json OpenVMM's outcome report of the latest start -//! ``` -//! -//! The CLI keeps its own record, the configuration, in its own formats; the store keeps the -//! runtime records, and reads no other format than its own: a sandbox that an earlier build -//! started must be stopped with that build. -//! -//! [`provision`], [`start`], [`stop`], and [`deprovision`] hold the [`LifecycleLock`] for their -//! whole transition, and [`exec_workload`] only while it finds the running VM, so a long workload -//! cannot delay a stop. `config.json` also records the binding of the distro layer that -//! `provision` verified, and that of the scratch image, which [`start`] checks from metadata -//! alone, and the network as its typed policy, which [`start`] checks again, in place of the -//! network options that `sandbox_lifecycle.py` recorded: it refuses to start a sandbox that -//! `nvx.py` or an earlier `nvx-rs` provisioned, which has no bindings or no typed network. It -//! records shares as the export plan that `provision` made, whose host objects [`start`] checks -//! again. [`start`] exports the shares, and once the guest agent is ready sends it the plan's -//! mapping table, which it mounts in order before any workload runs; a mapping that fails ends the -//! start, after the guest shuts down as a stop has it do. -//! -//! Every transition but provision finds the sandbox's VM with [`supervisor::reattach`], as an -//! orchestrator does after it restarts: it ends a start that was interrupted, holds a recorded -//! OpenVMM that still runs through its verified identity, and clears the runtime record of one -//! that exited without a stop, which the first transition to find it reports, or which a start -//! replaces. A VM that another runtime identity than the artifacts' started, as after the -//! artifacts were replaced, is never driven, since its guest may speak other contracts: [`stop`] -//! ends it by identity-verified forced termination, without a control session, and the other -//! transitions refuse it. [`start`] launches OpenVMM with [`supervisor::launch`], under the -//! runtime identity that the install metadata of the artifacts names, and [`stop`] asks the guest -//! of a VM of that identity to shut down, waits for OpenVMM to exit, and reports a VM that does -//! not, without ending it otherwise. -//! -//! [`start`] hands OpenVMM, through a [`PreparedLaunch`] of the checked files, the objects that -//! the checks found, rather than paths that something could replace meanwhile, and claims the -//! scratch image for this sandbox while OpenVMM runs. It holds the checked files until the guest -//! is ready, or until it has ended an OpenVMM whose start failed. - -use std::ffi::OsString; -use std::fs::OpenOptions; -use std::io::Write; -use std::path::{Path, PathBuf}; -use std::time::{Duration, Instant}; - -use serde_json::{Value, json}; - -use super::lock::LifecycleLock; -use super::state::{ - Config, RecordedDistro, RecordedScratch, hex, random_bytes, read_config, serialize_launch, - temporary_path, write_json, -}; -use super::{Artifacts, error_with_cause}; -use crate::host::cli::deadline::{Deadline, RETRY_INTERVAL}; -use crate::host::cli::json::{Object, pretty, read_object}; -use crate::host::cli::session::{EXIT_CATEGORIES, ExecRequest, ExecResult, Session}; -use crate::host::cli::{Exit, Result, check_interrupt, io_error, output}; -use crate::host::paths; -use crate::host::sandbox::distro::DISTRO_POLICY_REVISION; -use crate::host::sandbox::scratch::CLAIM_RELEASE_WAIT; -use crate::host::sandbox::{PreparedLaunch, SandboxLaunch, require_file}; -use crate::host::store::{ - IdentityPart, LaunchId, LockedStore, OUTCOME_NAME, RUNTIME_NAME, Recovery, SandboxStore, -}; -use crate::host::supervisor::{self, Launched, Reattached, StopOutcome, Vmm, VmmCommand}; -use crate::protocol::{CAPABILITY_LEN, GuestFeatures}; -use crate::{ErrorCode, ErrorKind}; - -/// The launch configuration, the CLI's own record beside those of the store. -pub(crate) const CONFIG_NAME: &str = "config.json"; -/// How often a caller retries a lifecycle lock that another caller holds, and the removal of a -/// state directory that open files still keep. -pub(crate) const LOCK_RETRY_INTERVAL: Duration = RETRY_INTERVAL; -/// The version of outcome reports. -pub(crate) const OUTCOME_SCHEMA_VERSION: u64 = 1; - -/// The options of a launch, besides the launch itself, that `provision` records. -pub(crate) struct ProvisionConfig<'a> { - /// `--hypervisor`, resolved. - pub(crate) hypervisor: &'a str, - /// `--memory-mib`. - pub(crate) memory_mib: u64, - /// `--cmdline`. - pub(crate) cmdline: &'a str, -} - -/// Returns the error for an I/O error that an operation on `path` met. -fn os_error(error: &std::io::Error, path: &Path) -> Exit { - io_error(ErrorCode::Io, error, path) -} - -/// Removes `path` unless it is missing. -fn unlink_missing_ok(path: &Path) -> Result { - match std::fs::remove_file(path) { - Err(error) if error.kind() != std::io::ErrorKind::NotFound => Err(os_error(&error, path)), - _ => Ok(()), - } -} - -/// Returns a builder of directories of `mode`, which Windows ignores. -fn dir_builder(mode: u32) -> std::fs::DirBuilder { - #[cfg_attr(not(unix), allow(unused_mut))] - let mut builder = std::fs::DirBuilder::new(); - #[cfg(unix)] - std::os::unix::fs::DirBuilderExt::mode(&mut builder, mode); - #[cfg(not(unix))] - let _ = mode; - builder -} - -/// Creates `path` and its missing parents: parents with the default mode, `path` with `mode`, and -/// no error if `path` is a directory already. -fn make_directories(path: &Path, mode: u32) -> Result { - match dir_builder(mode).create(path) { - Ok(()) => Ok(()), - Err(error) if error.kind() == std::io::ErrorKind::NotFound => match parent(path) { - Some(parent) => { - make_directories(parent, 0o777)?; - make_directories_once(path, mode) - } - None => Err(os_error(&error, path)), - }, - Err(_) if path.is_dir() => Ok(()), - Err(error) => Err(os_error(&error, path)), - } -} - -/// Creates `path` once its parent exists, with no error if it is a directory already. -fn make_directories_once(path: &Path, mode: u32) -> Result { - match dir_builder(mode).create(path) { - Ok(()) => Ok(()), - Err(_) if path.is_dir() => Ok(()), - Err(error) => Err(os_error(&error, path)), - } -} - -/// Returns the parent of `path`, or `None` for a root, or for a path of one relative component. -fn parent(path: &Path) -> Option<&Path> { - path.parent() - .filter(|parent| !parent.as_os_str().is_empty()) -} - -/// Creates the state directory at `path`, as provision does. An existing directory stays as it is, -/// and the store then refuses one that is not private to the current user; any other existing -/// path fails as creating a directory fails for a path that exists. A new directory is created as -/// the store creates it, private to the current user, below parents that are created as needed, -/// even when an existing one goes meanwhile: nothing else creates the directory itself. -pub(crate) fn create_state_directory(path: &Path) -> Result { - if path.is_dir() { - return Ok(()); - } - if std::fs::symlink_metadata(path).is_ok() { - return Err(os_error(&already_exists(), path)); - } - if let Some(parent) = parent(path) { - make_directories(parent, 0o777)?; - } - SandboxStore::create(path) - .map(drop) - .map_err(error_with_cause) -} - -/// Returns the error with which creating a directory fails for a path that exists. -fn already_exists() -> std::io::Error { - #[cfg(windows)] - let error = std::io::Error::from_raw_os_error(183); - #[cfg(unix)] - let error = std::io::Error::from_raw_os_error(nix::errno::Errno::EEXIST as i32); - #[cfg(not(any(windows, unix)))] - let error = std::io::Error::from(std::io::ErrorKind::AlreadyExists); - error -} - -/// Resolves the state directory at `path`, and creates it with `create`, as -/// `_prepare_state_directory` does. It must be a directory and not a symbolic link. -pub(crate) fn prepare_state_directory(path: &str, create: bool) -> Result { - let resolved = paths::resolve(Path::new(path)).map_err(Exit::from_os_error)?; - if create { - create_state_directory(&resolved)?; - } - if resolved.is_symlink() || !resolved.is_dir() { - return Err(Exit::error( - ErrorCode::SandboxNotProvisioned, - format!( - "sandbox state path is not a plain directory: {}", - resolved.display() - ), - )); - } - Ok(resolved) -} - -/// Returns the destination of an outcome report, as `_outcome_destination` does: in a plain -/// directory, with a file name, and new. -fn outcome_destination(path: &str) -> Result { - let candidate = - paths::normalized(&paths::absolute(Path::new(path)).map_err(Exit::from_os_error)?); - let parent = candidate.parent().unwrap_or(&candidate); - if parent.is_symlink() || !parent.is_dir() { - return Err(Exit::error( - ErrorCode::Unsupported, - format!( - "outcome report parent is not a plain directory: {}", - parent.display() - ), - )); - } - let Some(name) = candidate.file_name() else { - return Err(Exit::error( - ErrorCode::Unsupported, - "outcome report path has no filename", - )); - }; - let resolved = paths::resolve(parent) - .map_err(Exit::from_os_error)? - .join(name); - if std::fs::symlink_metadata(&resolved).is_ok() { - return Err(Exit::error( - ErrorCode::OutcomeReportExists, - format!("outcome report already exists: {}", resolved.display()), - )); - } - Ok(resolved) -} - -/// Checks the destination of an outcome report before a workload runs, as -/// `validate_outcome_destination` does. -pub(crate) fn validate_outcome_destination(path: &str) -> Result { - outcome_destination(path).map(drop) -} - -/// Writes the JSON `text` to a new file at `path`: through a temporary file of mode 0600, synced, -/// then linked to its name, which must not exist, with `\n` line ends on every host. -fn write_new_json(path: &str, text: &str) -> Result { - let resolved = outcome_destination(path)?; - let temporary = temporary_path(&resolved)?; - let mut options = OpenOptions::new(); - options.write(true).create_new(true); - #[cfg(unix)] - std::os::unix::fs::OpenOptionsExt::mode(&mut options, 0o600); - let mut file = match options.open(&temporary) { - Ok(file) => file, - Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists => { - return Err(Exit::error( - ErrorCode::Io, - "failed to reserve an outcome report staging file", - )); - } - Err(error) => return Err(os_error(&error, &temporary)), - }; - let written = (|| -> Result { - let text = format!("{text}\n"); - file.write_all(text.as_bytes()) - .and_then(|()| file.sync_all()) - .map_err(Exit::from_os_error)?; - match std::fs::hard_link(&temporary, &resolved) { - Ok(()) => Ok(()), - Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists => Err(Exit::error( - ErrorCode::OutcomeReportExists, - format!("outcome report already exists: {}", resolved.display()), - )), - Err(_) => Err(Exit::error( - ErrorCode::Io, - format!("failed to publish outcome report: {}", resolved.display()), - )), - } - })(); - drop(file); - let _ = std::fs::remove_file(&temporary); - written -} - -/// The sections of an OpenVMM outcome report that a reader requires. -const OUTCOME_SECTIONS: [&str; 3] = ["outcome", "network_policy", "teardown"]; - -/// Reads an OpenVMM outcome report, as [`read_object`] reads a JSON file: of schema version 1, -/// with `outcome`, `network_policy`, and `teardown` objects. -fn read_openvmm_outcome(path: &Path) -> Result { - let report = read_object( - path, - "OpenVMM outcome report", - "schema_version", - &[OUTCOME_SCHEMA_VERSION], - )?; - for name in OUTCOME_SECTIONS { - if !report.get(name).is_some_and(Value::is_object) { - return Err(Exit::error( - ErrorCode::Io, - format!( - "OpenVMM outcome report has an invalid {name} section: {}", - path.display() - ), - )); - } - } - Ok(report) -} - -/// Writes the outcome report of a managed workload: its category and status, with a random -/// operation ID, and nothing of its output. -pub(crate) fn write_exec_outcome(path: &str, result: &ExecResult) -> Result { - if !EXIT_CATEGORIES.contains(&result.category) { - return Err(Exit::error( - ErrorCode::Protocol, - "managed workload returned an unsupported outcome category", - )); - } - let operation_id = hex(&random_bytes::<16>()?); - let report = json!({ - "schema_version": OUTCOME_SCHEMA_VERSION, - "operation_id": operation_id, - "outcome": { - "operation": "exec", - "category": result.category, - "status_code": result.returncode, - }, - }); - write_new_json(path, &pretty(&report)) -} - -/// Records a launch and its distro layer, at the path at which it was verified, with the layer's -/// binding, in a new state directory, as `provision` does, with the binding of its scratch image, -/// at the path that it records for it. -pub(crate) fn provision( - state_path: &str, - launch: &SandboxLaunch, - distro: &RecordedDistro, - config: &ProvisionConfig<'_>, - timeout: f64, -) -> Result { - let state_dir = prepare_state_directory(state_path, true)?; - let config_path = state_dir.join(CONFIG_NAME); - let runtime_path = state_dir.join(RUNTIME_NAME); - let _lock = LifecycleLock::acquire(&state_dir, timeout, true)?; - if config_path.exists() || runtime_path.exists() { - return Err(Exit::error( - ErrorCode::SandboxProvisioned, - "sandbox is already provisioned", - )); - } - launch.validated()?; - let scratch = RecordedScratch::new(launch)?; - write_json( - &config_path, - &serialize_launch(launch, distro, &scratch, config)?, - ) -} - -/// What a transition finds of the sandbox's VM, through [`supervisor::reattach`]. -enum Found { - /// No VM is recorded, or a start that was interrupted is over. - Nothing, - /// The recorded OpenVMM exited without a stop, and its runtime record is cleared. - Exited, - /// The VM runs under the runtime identity of the artifacts. - Live(Vmm), - /// The VM runs under another runtime identity, which differs in these parts. - Mismatched(Vmm, Vec), -} - -/// Finds the sandbox's VM in the locked store, as an orchestrator does after it restarts within -/// the same host boot: with [`supervisor::reattach`], under the runtime identity of `artifacts`. -/// -/// Only a recorded launch needs that identity, so a sandbox that never started, or that stopped, -/// needs no install metadata. Fails for a record that the store does not read, such as one of an -/// earlier format, and when whether the VM runs cannot be determined. -fn find_vm(locked: &mut LockedStore, artifacts: &Artifacts<'_>) -> Result { - if !records_launch(locked)? { - return Ok(Found::Nothing); - } - let runtime = artifacts.runtime_identity()?; - Ok( - match supervisor::reattach(locked, &runtime).map_err(error_with_cause)? { - Reattached::Live(vmm) => Found::Live(vmm), - Reattached::Mismatched { vmm, differences } => Found::Mismatched(vmm, differences), - Reattached::Exited { - record: Some(_), .. - } => Found::Exited, - Reattached::Exited { record: None, .. } => Found::Nothing, - Reattached::Indeterminate(reason) => { - return Err(Exit::error( - ErrorCode::VmmIndeterminate, - format!( - "cannot determine whether the OpenVMM process of the sandbox runs: \ - {reason}" - ), - )); - } - }, - ) -} - -/// Returns whether the locked store records a launch of the sandbox, in its launch journal or its -/// runtime record, which alone needs the runtime identity of the artifacts. -fn records_launch(locked: &mut LockedStore) -> Result { - Ok(locked.journal().map_err(error_with_cause)?.is_some() - || locked.runtime().map_err(error_with_cause)?.is_some()) -} - -/// Returns the parts of a runtime identity that `differences` names, as messages list them. -fn differing_parts(differences: &[IdentityPart]) -> String { - let parts: Vec<&str> = differences.iter().map(|part| part.name()).collect(); - parts.join(", ") -} - -/// Returns the error of a transition other than stop on a VM that another runtime identity than -/// the artifacts' started, which differs in `differences`: the CLI does not drive it, since it may -/// speak other contracts, and only its stop ends it. -fn mismatched(vmm: &Vmm, differences: &[IdentityPart]) -> Exit { - Exit::error( - ErrorCode::RuntimeMismatch, - format!( - "sandbox runs under another runtime identity than these artifacts, which differ in \ - {}, so it is not driven: stop it to terminate its OpenVMM process {} without a \ - guest shutdown", - differing_parts(differences), - vmm.record().process().pid - ), - ) -} - -/// The running VM of a sandbox, and the capability with which its control session attaches. -struct Running { - /// The VM. - vmm: Vmm, - /// Its control capability. - capability: [u8; CAPABILITY_LEN], -} - -/// Returns the running VM of the sandbox in the locked store of `state_dir`, as exec requires it. -fn running_vm( - state_dir: &Path, - locked: &mut LockedStore, - artifacts: &Artifacts<'_>, -) -> Result { - let found = found_vm(locked, artifacts)?; - running(state_dir, locked, found) -} - -/// Finds the sandbox's VM as [`find_vm`] does, for an operation other than a start, which holds -/// no download lock of its own: while it reads the runtime identity of the artifacts, which only a -/// recorded launch needs, it holds the lock of their downloads, as [`Artifacts::hold`] says, so -/// that no download replaces them meanwhile. A sandbox that records no launch needs neither the -/// artifacts nor their root. -fn found_vm(locked: &mut LockedStore, artifacts: &Artifacts<'_>) -> Result { - if !records_launch(locked)? { - return Ok(Found::Nothing); - } - let _hold = artifacts.hold()?; - find_vm(locked, artifacts) -} - -/// Returns the running VM that `found` describes in the locked store of `state_dir`, as exec and -/// stop require it, with its control capability. -fn running(state_dir: &Path, locked: &mut LockedStore, found: Found) -> Result { - let vmm = match found { - Found::Live(vmm) => vmm, - Found::Nothing => { - // A directory without the configuration of a sandbox holds no provisioned one, which - // a start would report too, rather than one that is stopped. - require_config(state_dir)?; - return Err(Exit::error( - ErrorCode::SandboxNotRunning, - "sandbox is not running", - )); - } - Found::Exited => { - return Err(Exit::error( - ErrorCode::SandboxExited, - "sandbox runtime state is stale because the OpenVMM process is not running", - )); - } - Found::Mismatched(vmm, differences) => return Err(mismatched(&vmm, &differences)), - }; - let capability = locked.read_capability().map_err(error_with_cause)?; - Ok(Running { vmm, capability }) -} - -/// Connects to the control endpoint of `vmm`, and attaches with `capability`. -fn connect(vmm: &Vmm, capability: &[u8; CAPABILITY_LEN], timeout: f64) -> Result { - Session::connect( - vmm.endpoint(), - vmm.record().process().pid, - capability, - timeout, - ) -} - -/// Checks that `state_dir` holds the configuration of a provisioned sandbox: a directory without -/// it holds none, whatever else it holds. -fn require_config(state_dir: &Path) -> Result { - require_file(&state_dir.join(CONFIG_NAME), "sandbox configuration") - .map_err(|error| Exit::error(ErrorCode::SandboxNotProvisioned, error.message())) -} - -/// Starts a provisioned sandbox, as `start` does. -pub(crate) fn start(state_path: &str, artifacts: &Artifacts<'_>, timeout: f64) -> Result { - let state_dir = prepare_state_directory(state_path, false)?; - let mut lock = LifecycleLock::acquire(&state_dir, timeout, false)?; - start_locked(&state_dir, lock.locked(), artifacts, timeout) -} - -/// The control features that the layered sandbox root must provide for the requests of -/// `nvx-rs sandbox`. -pub(crate) const REQUIRED_FEATURES: GuestFeatures = GuestFeatures::LAYERED_SANDBOX; - -/// Starts the sandbox of a locked state directory, as `_start` does, through the launch journal -/// of its store. -fn start_locked( - state_dir: &Path, - locked: &mut LockedStore, - artifacts: &Artifacts<'_>, - timeout: f64, -) -> Result { - let config_path = state_dir.join(CONFIG_NAME); - require_config(state_dir)?; - let config = read_config(&config_path)?; - // The release of a checkout without artifacts is downloaded before the runtime identity of a - // recorded launch is read from their install metadata, and no download replaces them until - // the start ends, once OpenVMM has read them, so that the runtime identity that the start - // records is theirs. A configuration without a hypervisor is malformed, which reading it below - // reports. - let _hold = match config.get("hypervisor").and_then(Value::as_str) { - Some(hypervisor) => Some(artifacts.ensure(hypervisor)?), - None => None, - }; - match find_vm(locked, artifacts)? { - Found::Nothing | Found::Exited => {} - Found::Live(_) => { - return Err(Exit::error( - ErrorCode::SandboxRunning, - "sandbox is already running or has stale runtime state", - )); - } - Found::Mismatched(vmm, differences) => return Err(mismatched(&vmm, &differences)), - } - let outcome_path = state_dir.join(OUTCOME_NAME); - unlink_missing_ok(&outcome_path)?; - let config = Config::new(config)?; - let launch = config.launch()?; - // The checks read no content of the distro layer, its manifest, or the scratch image, and - // never verify them again. The prepared launch holds the files that they return, and hands - // OpenVMM those objects: on Linux, OpenVMM inherits their descriptors; on Windows, it opens - // them by their final paths, which the files keep anything from replacing before OpenVMM, - // whose opens share them, has opened them. The scratch image's check also claims it for this - // sandbox, until OpenVMM exits. - let binding = config.distro_binding()?; - let scratch_binding = config.scratch_binding()?; - let distro = binding - .check(launch.distro(), DISTRO_POLICY_REVISION) - .map_err(binding_error)?; - // Reading the network checks its policy again, as provision did, before anything launches. - let launch = launch.with_network(config.network()?); - let scratch = scratch_binding - .check(launch.scratch(), Instant::now() + CLAIM_RELEASE_WAIT) - .map_err(binding_error)?; - // The plan takes the format and identity of the distro layer from its binding. - let plan = launch.bound_plan(&binding)?; - let [executable, kernel, initrd] = artifacts.require()?; - let runtime = artifacts.launch_identity()?; - let capability = random_bytes::()?; - if capability == [0; CAPABILITY_LEN] { - return Err(Exit::error( - ErrorCode::Other, - "the system returned an all-zero control capability", - )); - } - let memory_mib = config.memory_mib(); - let prepared = PreparedLaunch::new(plan, distro, Some(scratch), config.cmdline())?; - // The control endpoint, which the launch chooses, goes between these arguments. - let mut head = vec![executable]; - head.extend(prepared.openvmm_arguments().iter().cloned()); - head.extend( - [ - "--microvm-lifecycle", - "managed", - "--single-process", - "--hypervisor", - ] - .map(str::to_owned), - ); - head.push(config.hypervisor().to_owned()); - head.push("--memory".to_owned()); - head.push(format!("{memory_mib}M")); - head.extend(["--kernel".to_owned(), kernel, "--initrd".to_owned(), initrd]); - head.push("--cmdline".to_owned()); - head.push(prepared.kernel_command_line().to_owned()); - head.extend(["--virtio-console", "none", "--microvm-control-console"].map(str::to_owned)); - let tail = vec![ - "--microvm-control-auth-stdin".to_owned(), - "--microvm-report".to_owned(), - outcome_path.display().to_string(), - ]; - let directory = std::env::current_dir().map_err(Exit::from_os_error)?; - let mut command = Vec::new(); - let launch = LaunchId::generate().map_err(error_with_cause)?; - let mut starting = supervisor::launch(locked, &launch, &runtime, &capability, |endpoint| { - command = head; - command.push(format!("listen={endpoint}")); - command.extend(tail); - let arguments = command[1..].iter().map(OsString::from).collect(); - let mut vmm = VmmCommand::new(&command[0], arguments, directory); - // OpenVMM inherits the checked files that the prepared launch passes, and nothing else. - vmm.inherit = prepared.inherited(); - Ok(vmm) - }) - .map_err(error_with_cause)?; - let pid = starting.process().reference().pid; - let maps = prepared.plan().mapping_table(); - // Whether the guest agent acknowledged a request to shut down after a mapping failed. - let mut stopping = false; - let ready = - check_ready(starting.endpoint(), pid, &capability, timeout).and_then(|mut session| { - // Workloads run only once the guest agent has mapped every share in order, so a - // mapping that fails ends the start. A sandbox without shares maps nothing. - if maps.is_empty() { - return Ok(()); - } - session.map_host_paths(&maps, timeout).inspect_err(|error| { - // The agent then refuses workloads, so the guest shuts down as a stop has it do: - // it unmounts what it mapped, the export, the overlay, and the scratch disk before - // the VM powers off. A failed transfer closed the session, so a guest that stopped - // answering is terminated at once. - if !matches!(error, Exit::Interrupted) { - stopping = session.stop(timeout).is_ok(); - } - }) - }); - let Err(error) = ready else { - // The guest is ready, so OpenVMM holds the images: the checked files can go, which - // releases their guards, while OpenVMM keeps the scratch image's claim. - drop(prepared); - return starting.commit().map(drop).map_err(error_with_cause); - }; - // OpenVMM closes the control endpoint as it tears the VM down, before it publishes its - // outcome report and exits, so only a closed endpoint, or a guest that shuts down, warrants - // waiting for it. - let grace = if stopping || matches!(error, Exit::EndpointClosed) { - timeout - } else { - 0.0 - }; - let status = end_failed_start(starting.child(), grace, &command)?; - // OpenVMM has exited, so the launch ends: its capability, control socket, report staging - // file, and journal go, and its log and outcome report stay. - let unended = match starting.abandon() { - Ok(Recovery::Indeterminate(reason)) => Some(reason), - Ok(_) => None, - Err(failure) => Some(failure.message().to_owned()), - }; - // The checked files can go too, which releases their guards and the scratch image's claim. - drop(prepared); - let error = match status { - // A guest that shut down after a mapping failed exits as a stopped one does, so the - // mapping's failure tells why the start failed. - Some(status) if !stopping && !matches!(error, Exit::Interrupted) => Exit::error( - ErrorCode::Exited, - startup_exit_message(&outcome_path, status)?, - ), - _ => error, - }; - if let Some(reason) = unended - && let Some(message) = error.message() - { - return Err(Exit::error( - error.code().unwrap_or(ErrorCode::Other), - format!("{message}; the launch could not be ended, and remains for recovery: {reason}"), - )); - } - Err(error) -} - -/// Returns the error of a start whose distro layer or scratch image fails the check of its -/// binding: a binding that no longer holds needs a new provision, which verifies the layer and -/// binds the image again, while a scratch image that another sandbox claims, and a failure of the -/// host, do not. The error keeps its code. -fn binding_error(error: crate::Error) -> Exit { - match error.kind() { - ErrorKind::Io | ErrorKind::TimedOut => error.into(), - _ => Exit::error( - error.code(), - format!( - "{}; deprovision the sandbox and provision it again", - error.message() - ), - ), - } -} - -/// Waits until the guest of a VM that starts answers on `endpoint`, which OpenVMM process `pid` -/// serves, checks that it provides the features of [`REQUIRED_FEATURES`], and returns the session, -/// on which the caller sends the guest what it needs before the launch commits. -fn check_ready( - endpoint: &str, - pid: u32, - capability: &[u8; CAPABILITY_LEN], - timeout: f64, -) -> Result { - let mut session = Session::connect(endpoint, pid, capability, timeout)?; - session.ping(timeout)?; - let features = session.features(timeout)?; - let missing = features.missing(REQUIRED_FEATURES); - if !missing.is_empty() { - return Err(Exit::error( - ErrorCode::GuestFeaturesMissing, - format!( - "the guest image lacks control features that layered sandboxes need ({}); use \ - an NVX release or build that includes them", - missing.join(", ") - ), - )); - } - Ok(session) -} - -/// Ends the OpenVMM process of a failed start, as `_end_failed_start` does: returns OpenVMM's -/// status if it exits on its own within `grace` seconds, or terminates it and returns `None`. -/// Terminating OpenVMM while it publishes its outcome report would lose the report, and leave -/// only its staging file, which the store removes as the launch ends. -fn end_failed_start( - launched: &mut Launched, - grace: f64, - command: &[String], -) -> Result> { - let waited = if grace > 0.0 { - wait_interruptibly(launched, Deadline::after(grace)) - } else { - Ok(()) - }; - let status = launched - .wait_timeout(Duration::ZERO) - .map_err(Exit::from_os_error)?; - if status.is_none() { - launched.terminate().map_err(Exit::from_os_error)?; - let exited = launched - .wait_timeout(Duration::from_secs(5)) - .map_err(Exit::from_os_error)?; - if exited.is_none() { - launched.kill().map_err(Exit::from_os_error)?; - let killed = launched - .wait_timeout(Duration::from_secs(5)) - .map_err(Exit::from_os_error)?; - if killed.is_none() { - return Err(Exit::error( - ErrorCode::TimedOut, - format!("command {command:?} did not exit within 5 seconds of its kill"), - )); - } - } - } - // Ctrl-C during the grace period ends the start as interrupted, after OpenVMM. - waited?; - Ok(status) -} - -/// Waits until `launched` exits or `deadline` passes, failing if Ctrl-C arrives. -fn wait_interruptibly(launched: &mut Launched, deadline: Deadline) -> Result { - loop { - let slice = deadline - .remaining() - .map_or(RETRY_INTERVAL, |left| left.min(RETRY_INTERVAL)); - crate::host::cli::check_interrupt()?; - if launched - .wait_timeout(slice) - .map_err(Exit::from_os_error)? - .is_some() - || deadline.passed() - { - return Ok(()); - } - } -} - -/// Describes how OpenVMM ended a start, preferring its outcome report, which keeps the guest's -/// own status: its category and its integer status, when it holds both. -fn startup_exit_message(outcome_path: &Path, status: i64) -> Result { - let report = read_openvmm_outcome(outcome_path).ok(); - if let Some(outcome) = report.as_ref().and_then(|report| report.get("outcome")) - && let Some(Value::String(category)) = outcome.get("category") - && let Some(status_code) = outcome.get("status_code").and_then(Value::as_i64) - { - return Ok(format!( - "OpenVMM exited during startup: {category} status {status_code}" - )); - } - Ok(format!( - "OpenVMM exited during startup with status {status}" - )) -} - -/// Runs a workload in a running sandbox, as `exec_workload` does. The lock is held only while the -/// running VM is found, so a long workload cannot delay a stop. -pub(crate) fn exec_workload( - state_path: &str, - artifacts: &Artifacts<'_>, - request: &ExecRequest, - response_timeout: f64, -) -> Result { - let state_dir = prepare_state_directory(state_path, false)?; - let running = { - let mut lock = LifecycleLock::acquire(&state_dir, response_timeout, false)?; - running_vm(&state_dir, lock.locked(), artifacts)? - }; - let mut session = connect(&running.vmm, &running.capability, response_timeout)?; - session.exec(request, response_timeout) -} - -/// Stops a running sandbox, and checks the outcome report that its OpenVMM wrote, unless another -/// runtime identity than the artifacts' started the VM. -/// -/// The supervisor's stop runs one graceful step, which asks the guest to shut the VM down and -/// waits up to `timeout` seconds for OpenVMM to exit, so that Ctrl-C ends the wait. A VM whose -/// OpenVMM is still running then is reported, and keeps its runtime record: the stop never takes -/// the forced step. -/// -/// A VM of another runtime identity may speak other contracts, so the stop asks it nothing: it -/// ends the VM as [`terminate_mismatched`] does. -pub(crate) fn stop(state_path: &str, artifacts: &Artifacts<'_>, timeout: f64) -> Result { - let state_dir = prepare_state_directory(state_path, false)?; - let mut lock = LifecycleLock::acquire(&state_dir, timeout, false)?; - let locked = lock.locked(); - let Running { vmm, capability } = match found_vm(locked, artifacts)? { - Found::Mismatched(vmm, differences) => { - return terminate_mismatched(locked, &vmm, &differences, timeout); - } - found => running(&state_dir, locked, found)?, - }; - // The graceful step waits for the exit itself, so the stop's own wait, until a deadline that - // has passed by then, only checks whether OpenVMM has exited. - let stopped = vmm - .stop(locked, Instant::now(), None, |vmm, _| { - { - let mut session = connect(vmm, &capability, timeout)?; - session.stop(timeout)?; - } - wait_for_exit(vmm, Deadline::after(timeout)).map(drop) - }) - .map_err(error_with_cause)?; - if let Some(error) = stopped.graceful_error { - return Err(error); - } - if stopped.outcome == StopOutcome::TimedOut { - return Err(Exit::error( - ErrorCode::TimedOut, - "OpenVMM did not terminate after managed stop", - )); - } - read_openvmm_outcome(&state_dir.join(OUTCOME_NAME)).map(drop) -} - -/// Ends the VM `vmm`, which another runtime identity than the artifacts' started, and which -/// differs from them in `differences`, by identity-verified forced termination, and warns on -/// standard error that it did. -/// -/// Its guest is not asked to shut down, and no control session is opened to it: its OpenVMM is -/// terminated through its verified process identity, as [`Vmm::terminate`] does, and the stop -/// waits up to `timeout` seconds for it to exit, as [`terminate_within`] does, so that Ctrl-C ends -/// the wait. One that does not exit then, whose exit cannot be determined, or whose wait Ctrl-C -/// ends, keeps its runtime record, so that a later stop retries. -fn terminate_mismatched( - locked: &mut LockedStore, - vmm: &Vmm, - differences: &[IdentityPart], - timeout: f64, -) -> Result { - let pid = vmm.record().process().pid; - let parts = differing_parts(differences); - let outcome = terminate_within( - Deadline::after(timeout), - |bound| vmm.terminate(locked, bound).map_err(error_with_cause), - |deadline| wait_for_exit(vmm, deadline), - )?; - let ended = match outcome { - StopOutcome::Forced => "was terminated without a guest shutdown", - StopOutcome::Graceful => "had exited", - StopOutcome::TimedOut => { - return Err(Exit::error( - ErrorCode::TimedOut, - format!( - "OpenVMM process {pid} of the sandbox, which runs under another runtime \ - identity than these artifacts, did not exit after it was terminated" - ), - )); - } - }; - output::print_stderr(&format!( - "warning: sandbox ran under another runtime identity than these artifacts, which differ \ - in {parts}: its OpenVMM process {pid} {ended}" - )); - Ok(()) -} - -/// Ends an OpenVMM by identity-verified forced termination, and waits until `deadline` for it to -/// exit: `terminate` terminates it unless it has exited, waits up to the bound that it takes for -/// it to exit, and clears its runtime record once it has, as [`Vmm::terminate`] does, and `wait` -/// waits for its exit until a deadline, fails if Ctrl-C arrives first, and returns whether it -/// exited, as [`wait_for_exit`] does. -/// -/// The termination waits for one retry interval at most, and `wait` for the rest, so that Ctrl-C -/// ends the wait for an OpenVMM that does not exit at once, as one in uninterruptible I/O. Once -/// that one has exited, a second termination finds it exited, signals nothing, and clears its -/// record, and the result is [`StopOutcome::Forced`] all the same. One that still runs at the -/// deadline is [`StopOutcome::TimedOut`], and Ctrl-C fails the termination: both keep the record. -fn terminate_within( - deadline: Deadline, - mut terminate: impl FnMut(Duration) -> Result, - wait: impl FnOnce(Deadline) -> Result, -) -> Result { - check_interrupt()?; - // A termination's bound is positive, though the deadline may have passed. - let bound = deadline - .remaining() - .map_or(RETRY_INTERVAL, |left| left.min(RETRY_INTERVAL)) - .max(Duration::from_nanos(1)); - match terminate(bound)? { - StopOutcome::TimedOut if wait(deadline)? => match terminate(bound)? { - StopOutcome::TimedOut => Ok(StopOutcome::TimedOut), - StopOutcome::Graceful | StopOutcome::Forced => Ok(StopOutcome::Forced), - }, - outcome => Ok(outcome), - } -} - -/// Waits until the OpenVMM of `vmm` exits or `deadline` passes, and returns whether it exited, -/// failing if Ctrl-C arrives first. -fn wait_for_exit(vmm: &Vmm, deadline: Deadline) -> Result { - loop { - check_interrupt()?; - let slice = deadline - .remaining() - .map_or(RETRY_INTERVAL, |left| left.min(RETRY_INTERVAL)); - if vmm - .wait(Instant::now() + slice) - .map_err(Exit::from_os_error)? - { - return Ok(true); - } - if deadline.passed() { - return Ok(false); - } - } -} - -/// Removes a stopped sandbox and its state directory, as `deprovision` does. -pub(crate) fn deprovision(state_path: &str, artifacts: &Artifacts<'_>, timeout: f64) -> Result { - let state_dir = prepare_state_directory(state_path, false)?; - let mut lock = LifecycleLock::acquire(&state_dir, timeout, false)?; - match found_vm(lock.locked(), artifacts)? { - Found::Nothing | Found::Exited => {} - Found::Live(_) | Found::Mismatched(..) => { - return Err(Exit::error( - ErrorCode::SandboxRunning, - "sandbox must be stopped before deprovision", - )); - } - } - // No OpenVMM of the sandbox runs, and a start that was interrupted is over, so the store - // removes the sandbox: its files and the configuration, then refuses files that are not - // NVX's, and otherwise removes its lock file, while it still holds the lock, and the - // directory. - lock.remove(timeout) -} - -#[cfg(test)] -mod tests { - use std::cell::{Cell, RefCell}; - - use super::*; - - /// An OpenVMM of another runtime identity whose termination a test scripts, without a process. - struct Scripted { - /// Whether OpenVMM has exited. - exited: Cell, - /// Whether a termination ends OpenVMM within its bound, which one in uninterruptible I/O - /// outlives. - dies_when_terminated: bool, - /// Whether the runtime record of its launch remains. - recorded: Cell, - /// The bounds of its terminations, in order. - bounds: RefCell>, - } - - impl Scripted { - /// Returns an OpenVMM that runs, which a termination ends if `dies_when_terminated`. - fn running(dies_when_terminated: bool) -> Self { - Self { - exited: Cell::new(false), - dies_when_terminated, - recorded: Cell::new(true), - bounds: RefCell::new(Vec::new()), - } - } - - /// Terminates OpenVMM within `bound`, as [`Vmm::terminate`] does: one that has exited is - /// not signaled, and the runtime record goes once it has exited. - fn terminate(&self, bound: Duration) -> Result { - self.bounds.borrow_mut().push(bound); - let outcome = if self.exited.get() { - StopOutcome::Graceful - } else if self.dies_when_terminated { - self.exited.set(true); - StopOutcome::Forced - } else { - StopOutcome::TimedOut - }; - if self.exited.get() { - self.recorded.set(false); - } - Ok(outcome) - } - - /// Ends OpenVMM, as a stop with an hour's timeout does, and ends the stop's wait for its - /// exit with `waited`: OpenVMM exits during the wait if that is `Ok(true)`. - fn stop(&self, waited: Result) -> Result { - terminate_within( - Deadline::after(3600.0), - |bound| self.terminate(bound), - |_| { - if matches!(waited, Ok(true)) { - self.exited.set(true); - } - waited - }, - ) - } - - /// Returns the bounds of its terminations. - fn bounds(&self) -> Vec { - self.bounds.borrow().clone() - } - } - - #[test] - fn ctrl_c_ends_the_wait_for_a_terminated_openvmm_that_has_not_exited() { - // OpenVMM outlives its termination, so the termination waits one retry interval at most, - // whatever the timeout, and the stop's own wait, which Ctrl-C ends, waits for the rest. - let openvmm = Scripted::running(false); - let stopped = openvmm.stop(Err(Exit::Interrupted)); - assert!(matches!(stopped, Err(Exit::Interrupted)), "{stopped:?}"); - let bounds = openvmm.bounds(); - assert_eq!(bounds.len(), 1, "{bounds:?}"); - assert!(bounds[0] <= RETRY_INTERVAL, "{bounds:?}"); - // Its exit was not seen, so its runtime record remains for the next stop. - assert!(openvmm.recorded.get()); - } - - #[test] - fn a_terminated_openvmm_that_exits_during_the_wait_was_forced() { - // Once it has exited, a second termination finds it exited, signals nothing, and clears - // its runtime record. - let openvmm = Scripted::running(false); - let stopped = openvmm.stop(Ok(true)); - assert!(matches!(stopped, Ok(StopOutcome::Forced)), "{stopped:?}"); - assert_eq!(openvmm.bounds().len(), 2); - assert!(!openvmm.recorded.get()); - - // One that still runs at the deadline keeps its record. - let survivor = Scripted::running(false); - let stopped = survivor.stop(Ok(false)); - assert!(matches!(stopped, Ok(StopOutcome::TimedOut)), "{stopped:?}"); - assert_eq!(survivor.bounds().len(), 1); - assert!(survivor.recorded.get()); - } - - #[test] - fn an_openvmm_that_its_termination_ends_or_that_exited_is_not_waited_for() { - // The stop's wait, which would fail here, does not run. - let openvmm = Scripted::running(true); - let stopped = openvmm.stop(Err(Exit::Interrupted)); - assert!(matches!(stopped, Ok(StopOutcome::Forced)), "{stopped:?}"); - assert_eq!(openvmm.bounds().len(), 1); - assert!(!openvmm.recorded.get()); - - let exited = Scripted::running(true); - exited.exited.set(true); - let stopped = exited.stop(Err(Exit::Interrupted)); - assert!(matches!(stopped, Ok(StopOutcome::Graceful)), "{stopped:?}"); - assert_eq!(exited.bounds().len(), 1); - assert!(!exited.recorded.get()); - } -} diff --git a/nvx-rs/src/host/cli/sandbox/lifecycle/directory.rs b/nvx-rs/src/host/cli/sandbox/lifecycle/directory.rs new file mode 100644 index 000000000..3fa5ecd95 --- /dev/null +++ b/nvx-rs/src/host/cli/sandbox/lifecycle/directory.rs @@ -0,0 +1,115 @@ +//! Creation and resolution of a sandbox's private state directory. + +use std::path::{Path, PathBuf}; + +use crate::ErrorCode; +use crate::host::cli::sandbox::error_with_cause; +use crate::host::cli::{Exit, Result, io_error}; +use crate::host::paths; +use crate::host::store::SandboxStore; + +/// Returns the error for an I/O error that an operation on `path` met. +pub(super) fn os_error(error: &std::io::Error, path: &Path) -> Exit { + io_error(ErrorCode::Io, error, path) +} + +/// Removes `path` unless it is missing. +pub(super) fn unlink_missing_ok(path: &Path) -> Result { + match std::fs::remove_file(path) { + Err(error) if error.kind() != std::io::ErrorKind::NotFound => Err(os_error(&error, path)), + _ => Ok(()), + } +} + +/// Returns a builder of directories of `mode`, which Windows ignores. +fn dir_builder(mode: u32) -> std::fs::DirBuilder { + #[cfg_attr(not(unix), allow(unused_mut))] + let mut builder = std::fs::DirBuilder::new(); + #[cfg(unix)] + std::os::unix::fs::DirBuilderExt::mode(&mut builder, mode); + #[cfg(not(unix))] + let _ = mode; + builder +} + +/// Creates `path` and its missing parents: parents with the default mode, `path` with `mode`, and +/// no error if `path` is a directory already. +fn make_directories(path: &Path, mode: u32) -> Result { + match dir_builder(mode).create(path) { + Ok(()) => Ok(()), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => match parent(path) { + Some(parent) => { + make_directories(parent, 0o777)?; + make_directories_once(path, mode) + } + None => Err(os_error(&error, path)), + }, + Err(_) if path.is_dir() => Ok(()), + Err(error) => Err(os_error(&error, path)), + } +} + +/// Creates `path` once its parent exists, with no error if it is a directory already. +fn make_directories_once(path: &Path, mode: u32) -> Result { + match dir_builder(mode).create(path) { + Ok(()) => Ok(()), + Err(_) if path.is_dir() => Ok(()), + Err(error) => Err(os_error(&error, path)), + } +} + +/// Returns the parent of `path`, or `None` for a root, or for a path of one relative component. +fn parent(path: &Path) -> Option<&Path> { + path.parent() + .filter(|parent| !parent.as_os_str().is_empty()) +} + +/// Creates the state directory at `path`, as provision does. An existing directory stays as it is, +/// and the store then refuses one that is not private to the current user; any other existing +/// path fails as creating a directory fails for a path that exists. A new directory is created as +/// the store creates it, private to the current user, below parents that are created as needed, +/// even when an existing one goes meanwhile: nothing else creates the directory itself. +pub(crate) fn create_state_directory(path: &Path) -> Result { + if path.is_dir() { + return Ok(()); + } + if std::fs::symlink_metadata(path).is_ok() { + return Err(os_error(&already_exists(), path)); + } + if let Some(parent) = parent(path) { + make_directories(parent, 0o777)?; + } + SandboxStore::create(path) + .map(drop) + .map_err(error_with_cause) +} + +/// Returns the error with which creating a directory fails for a path that exists. +fn already_exists() -> std::io::Error { + #[cfg(windows)] + let error = std::io::Error::from_raw_os_error(183); + #[cfg(unix)] + let error = std::io::Error::from_raw_os_error(nix::errno::Errno::EEXIST as i32); + #[cfg(not(any(windows, unix)))] + let error = std::io::Error::from(std::io::ErrorKind::AlreadyExists); + error +} + +/// Resolves the state directory at `path`, and creates it with `create`, as +/// `_prepare_state_directory` does. It must be a directory and not a symbolic link. +pub(crate) fn prepare_state_directory(path: &str, create: bool) -> Result { + let resolved = paths::resolve(Path::new(path)).map_err(Exit::from_os_error)?; + if create { + create_state_directory(&resolved)?; + } + if resolved.is_symlink() || !resolved.is_dir() { + return Err(Exit::error( + ErrorCode::SandboxNotProvisioned, + format!( + "sandbox state path is not a plain directory: {}", + resolved.display() + ), + )); + } + Ok(resolved) +} diff --git a/nvx-rs/src/host/cli/sandbox/lifecycle/mod.rs b/nvx-rs/src/host/cli/sandbox/lifecycle/mod.rs new file mode 100644 index 000000000..430d1c152 --- /dev/null +++ b/nvx-rs/src/host/cli/sandbox/lifecycle/mod.rs @@ -0,0 +1,280 @@ +//! The lifecycle of a managed sandbox, a port of the former +//! `scripts/nvx_tools/sandbox_lifecycle.py`, which it replaced (microsoft/nvx#359), on the +//! [`store`](crate::host::store), [`supervisor`], and [`session`](crate::host::session) that +//! every host of a sandbox shares. +//! +//! A sandbox lives in its state directory, which is the directory of its store: +//! +//! ```text +//! / +//! config.json the launch that provision records, in format 7, with the export plan of +//! its shares or `null` without shares, the bindings of its distro layer +//! and of its scratch image, and its network +//! sandbox.lock the store's lock, which serializes the lifecycle transitions +//! launch.json the store's journal of a start in progress, or of an interrupted one +//! runtime.json the store's runtime record of the running VM, in its format 4 +//! control.capability the 32-byte capability of the running OpenVMM +//! control.sock the control endpoint of the running OpenVMM, on POSIX +//! openvmm.log OpenVMM's output, which each start replaces +//! outcome.json OpenVMM's outcome report of the latest start +//! ``` +//! +//! The CLI keeps its own record, the configuration, in its own formats; the store keeps the +//! runtime records, and reads no other format than its own: a sandbox that an earlier build +//! started must be stopped with that build. +//! +//! [`provision`], [`fn@crate::host::cli::sandbox::lifecycle::start`], [`fn@crate::host::cli::sandbox::lifecycle::stop`], and [`deprovision`] hold the [`LifecycleLock`] for their +//! whole transition, and [`exec_workload`] only while it finds the running VM, so a long workload +//! cannot delay a stop. `config.json` also records the binding of the distro layer that +//! `provision` verified, and that of the scratch image, which [`fn@crate::host::cli::sandbox::lifecycle::start`] checks from metadata +//! alone, and the network as its typed policy, which [`fn@crate::host::cli::sandbox::lifecycle::start`] checks again, in place of the +//! network options that `sandbox_lifecycle.py` recorded: it refuses to start a sandbox that +//! `nvx.py` or an earlier `nvx-rs` provisioned, which has no bindings or no typed network. It +//! records shares as the export plan that `provision` made, whose host objects [`fn@crate::host::cli::sandbox::lifecycle::start`] checks +//! again. [`fn@crate::host::cli::sandbox::lifecycle::start`] exports the shares, and once the guest agent is ready sends it the plan's +//! mapping table, which it mounts in order before any workload runs; a mapping that fails ends the +//! start, after the guest shuts down as a stop has it do. +//! +//! Every transition but provision finds the sandbox's VM with [`supervisor::reattach`], as an +//! orchestrator does after it restarts: it ends a start that was interrupted, holds a recorded +//! OpenVMM that still runs through its verified identity, and clears the runtime record of one +//! that exited without a stop, which the first transition to find it reports, or which a start +//! replaces. A VM that another runtime identity than the artifacts' started, as after the +//! artifacts were replaced, is never driven, since its guest may speak other contracts: [`fn@crate::host::cli::sandbox::lifecycle::stop`] +//! ends it by identity-verified forced termination, without a control session, and the other +//! transitions refuse it. [`fn@crate::host::cli::sandbox::lifecycle::start`] launches OpenVMM with [`supervisor::launch`], under the +//! runtime identity that the install metadata of the artifacts names, and [`fn@crate::host::cli::sandbox::lifecycle::stop`] asks the guest +//! of a VM of that identity to shut down, waits for OpenVMM to exit, and reports a VM that does +//! not, without ending it otherwise. +//! +//! [`fn@crate::host::cli::sandbox::lifecycle::start`] hands OpenVMM, through a [`crate::host::sandbox::PreparedLaunch`] of the checked files, the objects that +//! the checks found, rather than paths that something could replace meanwhile, and claims the +//! scratch image for this sandbox while OpenVMM runs. It holds the checked files until the guest +//! is ready, or until it has ended an OpenVMM whose start failed. + +mod directory; +mod outcome; +mod start; +mod stop; + +use std::path::Path; +use std::time::Duration; + +use crate::ErrorCode; +use crate::host::cli::deadline::RETRY_INTERVAL; +use crate::host::cli::sandbox::lock::LifecycleLock; +use crate::host::cli::sandbox::state::{ + RecordedDistro, RecordedScratch, serialize_launch, write_json, +}; +use crate::host::cli::sandbox::{Artifacts, error_with_cause}; +use crate::host::cli::session::Session; +use crate::host::cli::{Exit, Result}; +use crate::host::sandbox::{SandboxLaunch, require_file}; +use crate::host::store::{IdentityPart, LockedStore, RUNTIME_NAME}; +use crate::host::supervisor::{self, Reattached, Vmm}; +use crate::protocol::{CAPABILITY_LEN, GuestFeatures}; + +pub(crate) use self::directory::{create_state_directory, prepare_state_directory}; +pub(crate) use self::outcome::{validate_outcome_destination, write_exec_outcome}; +pub(crate) use self::start::{exec_workload, start}; +pub(crate) use self::stop::{deprovision, stop}; + +/// The launch configuration, the CLI's own record beside those of the store. +pub(crate) const CONFIG_NAME: &str = "config.json"; +/// How often a caller retries a lifecycle lock that another caller holds, and the removal of a +/// state directory that open files still keep. +pub(crate) const LOCK_RETRY_INTERVAL: Duration = RETRY_INTERVAL; +/// The version of outcome reports. +pub(crate) const OUTCOME_SCHEMA_VERSION: u64 = 1; + +/// The options of a launch, besides the launch itself, that `provision` records. +pub(crate) struct ProvisionConfig<'a> { + /// `--hypervisor`, resolved. + pub(crate) hypervisor: &'a str, + /// `--memory-mib`. + pub(crate) memory_mib: u64, + /// `--cmdline`. + pub(crate) cmdline: &'a str, +} + +/// The control features that the layered sandbox root must provide for the requests of +/// `nvx-rs sandbox`. +pub(crate) const REQUIRED_FEATURES: GuestFeatures = GuestFeatures::LAYERED_SANDBOX; + +/// Records a launch and its distro layer, at the path at which it was verified, with the layer's +/// binding, in a new state directory, as `provision` does, with the binding of its scratch image, +/// at the path that it records for it. +pub(crate) fn provision( + state_path: &str, + launch: &SandboxLaunch, + distro: &RecordedDistro, + config: &ProvisionConfig<'_>, + timeout: f64, +) -> Result { + let state_dir = prepare_state_directory(state_path, true)?; + let config_path = state_dir.join(CONFIG_NAME); + let runtime_path = state_dir.join(RUNTIME_NAME); + let _lock = LifecycleLock::acquire(&state_dir, timeout, true)?; + if config_path.exists() || runtime_path.exists() { + return Err(Exit::error( + ErrorCode::SandboxProvisioned, + "sandbox is already provisioned", + )); + } + launch.validated()?; + let scratch = RecordedScratch::new(launch)?; + write_json( + &config_path, + &serialize_launch(launch, distro, &scratch, config)?, + ) +} + +/// What a transition finds of the sandbox's VM, through [`supervisor::reattach`]. +enum Found { + /// No VM is recorded, or a start that was interrupted is over. + Nothing, + /// The recorded OpenVMM exited without a stop, and its runtime record is cleared. + Exited, + /// The VM runs under the runtime identity of the artifacts. + Live(Vmm), + /// The VM runs under another runtime identity, which differs in these parts. + Mismatched(Vmm, Vec), +} + +/// Finds the sandbox's VM in the locked store, as an orchestrator does after it restarts within +/// the same host boot: with [`supervisor::reattach`], under the runtime identity of `artifacts`. +/// +/// Only a recorded launch needs that identity, so a sandbox that never started, or that stopped, +/// needs no install metadata. Fails for a record that the store does not read, such as one of an +/// earlier format, and when whether the VM runs cannot be determined. +fn find_vm(locked: &mut LockedStore, artifacts: &Artifacts<'_>) -> Result { + if !records_launch(locked)? { + return Ok(Found::Nothing); + } + let runtime = artifacts.runtime_identity()?; + Ok( + match supervisor::reattach(locked, &runtime).map_err(error_with_cause)? { + Reattached::Live(vmm) => Found::Live(vmm), + Reattached::Mismatched { vmm, differences } => Found::Mismatched(vmm, differences), + Reattached::Exited { + record: Some(_), .. + } => Found::Exited, + Reattached::Exited { record: None, .. } => Found::Nothing, + Reattached::Indeterminate(reason) => { + return Err(Exit::error( + ErrorCode::VmmIndeterminate, + format!( + "cannot determine whether the OpenVMM process of the sandbox runs: \ + {reason}" + ), + )); + } + }, + ) +} + +/// Returns whether the locked store records a launch of the sandbox, in its launch journal or its +/// runtime record, which alone needs the runtime identity of the artifacts. +fn records_launch(locked: &mut LockedStore) -> Result { + Ok(locked.journal().map_err(error_with_cause)?.is_some() + || locked.runtime().map_err(error_with_cause)?.is_some()) +} + +/// Returns the parts of a runtime identity that `differences` names, as messages list them. +fn differing_parts(differences: &[IdentityPart]) -> String { + let parts: Vec<&str> = differences.iter().map(|part| part.name()).collect(); + parts.join(", ") +} + +/// Returns the error of a transition other than stop on a VM that another runtime identity than +/// the artifacts' started, which differs in `differences`: the CLI does not drive it, since it may +/// speak other contracts, and only its stop ends it. +fn mismatched(vmm: &Vmm, differences: &[IdentityPart]) -> Exit { + Exit::error( + ErrorCode::RuntimeMismatch, + format!( + "sandbox runs under another runtime identity than these artifacts, which differ in \ + {}, so it is not driven: stop it to terminate its OpenVMM process {} without a \ + guest shutdown", + differing_parts(differences), + vmm.record().process().pid + ), + ) +} + +/// The running VM of a sandbox, and the capability with which its control session attaches. +struct Running { + /// The VM. + vmm: Vmm, + /// Its control capability. + capability: [u8; CAPABILITY_LEN], +} + +/// Returns the running VM of the sandbox in the locked store of `state_dir`, as exec requires it. +fn running_vm( + state_dir: &Path, + locked: &mut LockedStore, + artifacts: &Artifacts<'_>, +) -> Result { + let found = found_vm(locked, artifacts)?; + running(state_dir, locked, found) +} + +/// Finds the sandbox's VM as [`find_vm`] does, for an operation other than a start, which holds +/// no download lock of its own: while it reads the runtime identity of the artifacts, which only a +/// recorded launch needs, it holds the lock of their downloads, as [`Artifacts::hold`] says, so +/// that no download replaces them meanwhile. A sandbox that records no launch needs neither the +/// artifacts nor their root. +fn found_vm(locked: &mut LockedStore, artifacts: &Artifacts<'_>) -> Result { + if !records_launch(locked)? { + return Ok(Found::Nothing); + } + let _hold = artifacts.hold()?; + find_vm(locked, artifacts) +} + +/// Returns the running VM that `found` describes in the locked store of `state_dir`, as exec and +/// stop require it, with its control capability. +fn running(state_dir: &Path, locked: &mut LockedStore, found: Found) -> Result { + let vmm = match found { + Found::Live(vmm) => vmm, + Found::Nothing => { + // A directory without the configuration of a sandbox holds no provisioned one, which + // a start would report too, rather than one that is stopped. + require_config(state_dir)?; + return Err(Exit::error( + ErrorCode::SandboxNotRunning, + "sandbox is not running", + )); + } + Found::Exited => { + return Err(Exit::error( + ErrorCode::SandboxExited, + "sandbox runtime state is stale because the OpenVMM process is not running", + )); + } + Found::Mismatched(vmm, differences) => return Err(mismatched(&vmm, &differences)), + }; + let capability = locked.read_capability().map_err(error_with_cause)?; + Ok(Running { vmm, capability }) +} + +/// Connects to the control endpoint of `vmm`, and attaches with `capability`. +fn connect(vmm: &Vmm, capability: &[u8; CAPABILITY_LEN], timeout: f64) -> Result { + Session::connect( + vmm.endpoint(), + vmm.record().process().pid, + capability, + timeout, + ) +} + +/// Checks that `state_dir` holds the configuration of a provisioned sandbox: a directory without +/// it holds none, whatever else it holds. +fn require_config(state_dir: &Path) -> Result { + require_file(&state_dir.join(CONFIG_NAME), "sandbox configuration") + .map_err(|error| Exit::error(ErrorCode::SandboxNotProvisioned, error.message())) +} + +#[cfg(test)] +mod tests; diff --git a/nvx-rs/src/host/cli/sandbox/lifecycle/outcome.rs b/nvx-rs/src/host/cli/sandbox/lifecycle/outcome.rs new file mode 100644 index 000000000..a50cee787 --- /dev/null +++ b/nvx-rs/src/host/cli/sandbox/lifecycle/outcome.rs @@ -0,0 +1,144 @@ +//! Publication and reading of managed-workload and OpenVMM outcome reports. + +use std::fs::OpenOptions; +use std::io::Write; +use std::path::{Path, PathBuf}; + +use serde_json::{Value, json}; + +use super::OUTCOME_SCHEMA_VERSION; +use super::directory::os_error; +use crate::ErrorCode; +use crate::host::cli::json::{Object, pretty, read_object}; +use crate::host::cli::sandbox::state::{hex, random_bytes, temporary_path}; +use crate::host::cli::session::{EXIT_CATEGORIES, ExecResult}; +use crate::host::cli::{Exit, Result}; +use crate::host::paths; + +/// Returns the destination of an outcome report, as `_outcome_destination` does: in a plain +/// directory, with a file name, and new. +fn outcome_destination(path: &str) -> Result { + let candidate = + paths::normalized(&paths::absolute(Path::new(path)).map_err(Exit::from_os_error)?); + let parent = candidate.parent().unwrap_or(&candidate); + if parent.is_symlink() || !parent.is_dir() { + return Err(Exit::error( + ErrorCode::Unsupported, + format!( + "outcome report parent is not a plain directory: {}", + parent.display() + ), + )); + } + let Some(name) = candidate.file_name() else { + return Err(Exit::error( + ErrorCode::Unsupported, + "outcome report path has no filename", + )); + }; + let resolved = paths::resolve(parent) + .map_err(Exit::from_os_error)? + .join(name); + if std::fs::symlink_metadata(&resolved).is_ok() { + return Err(Exit::error( + ErrorCode::OutcomeReportExists, + format!("outcome report already exists: {}", resolved.display()), + )); + } + Ok(resolved) +} + +/// Checks the destination of an outcome report before a workload runs, as +/// `validate_outcome_destination` does. +pub(crate) fn validate_outcome_destination(path: &str) -> Result { + outcome_destination(path).map(drop) +} + +/// Writes the JSON `text` to a new file at `path`: through a temporary file of mode 0600, synced, +/// then linked to its name, which must not exist, with `\n` line ends on every host. +fn write_new_json(path: &str, text: &str) -> Result { + let resolved = outcome_destination(path)?; + let temporary = temporary_path(&resolved)?; + let mut options = OpenOptions::new(); + options.write(true).create_new(true); + #[cfg(unix)] + std::os::unix::fs::OpenOptionsExt::mode(&mut options, 0o600); + let mut file = match options.open(&temporary) { + Ok(file) => file, + Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists => { + return Err(Exit::error( + ErrorCode::Io, + "failed to reserve an outcome report staging file", + )); + } + Err(error) => return Err(os_error(&error, &temporary)), + }; + let written = (|| -> Result { + let text = format!("{text}\n"); + file.write_all(text.as_bytes()) + .and_then(|()| file.sync_all()) + .map_err(Exit::from_os_error)?; + match std::fs::hard_link(&temporary, &resolved) { + Ok(()) => Ok(()), + Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists => Err(Exit::error( + ErrorCode::OutcomeReportExists, + format!("outcome report already exists: {}", resolved.display()), + )), + Err(_) => Err(Exit::error( + ErrorCode::Io, + format!("failed to publish outcome report: {}", resolved.display()), + )), + } + })(); + drop(file); + let _ = std::fs::remove_file(&temporary); + written +} + +/// The sections of an OpenVMM outcome report that a reader requires. +const OUTCOME_SECTIONS: [&str; 3] = ["outcome", "network_policy", "teardown"]; + +/// Reads an OpenVMM outcome report, as [`read_object`] reads a JSON file: of schema version 1, +/// with `outcome`, `network_policy`, and `teardown` objects. +pub(super) fn read_openvmm_outcome(path: &Path) -> Result { + let report = read_object( + path, + "OpenVMM outcome report", + "schema_version", + &[OUTCOME_SCHEMA_VERSION], + )?; + for name in OUTCOME_SECTIONS { + if !report.get(name).is_some_and(Value::is_object) { + return Err(Exit::error( + ErrorCode::Io, + format!( + "OpenVMM outcome report has an invalid {name} section: {}", + path.display() + ), + )); + } + } + Ok(report) +} + +/// Writes the outcome report of a managed workload: its category and status, with a random +/// operation ID, and nothing of its output. +pub(crate) fn write_exec_outcome(path: &str, result: &ExecResult) -> Result { + if !EXIT_CATEGORIES.contains(&result.category) { + return Err(Exit::error( + ErrorCode::Protocol, + "managed workload returned an unsupported outcome category", + )); + } + let operation_id = hex(&random_bytes::<16>()?); + let report = json!({ + "schema_version": OUTCOME_SCHEMA_VERSION, + "operation_id": operation_id, + "outcome": { + "operation": "exec", + "category": result.category, + "status_code": result.returncode, + }, + }); + write_new_json(path, &pretty(&report)) +} diff --git a/nvx-rs/src/host/cli/sandbox/lifecycle/start.rs b/nvx-rs/src/host/cli/sandbox/lifecycle/start.rs new file mode 100644 index 000000000..e8219c574 --- /dev/null +++ b/nvx-rs/src/host/cli/sandbox/lifecycle/start.rs @@ -0,0 +1,334 @@ +//! Starting a managed sandbox and executing workloads in its running VM. + +use std::ffi::OsString; +use std::path::Path; +use std::time::{Duration, Instant}; + +use serde_json::Value; + +use super::directory::{prepare_state_directory, unlink_missing_ok}; +use super::outcome::read_openvmm_outcome; +use super::{ + CONFIG_NAME, Found, REQUIRED_FEATURES, connect, find_vm, mismatched, require_config, running_vm, +}; +use crate::host::cli::deadline::{Deadline, RETRY_INTERVAL}; +use crate::host::cli::sandbox::lock::LifecycleLock; +use crate::host::cli::sandbox::state::{Config, random_bytes, read_config}; +use crate::host::cli::sandbox::{Artifacts, error_with_cause}; +use crate::host::cli::session::{ExecRequest, ExecResult, Session}; +use crate::host::cli::{Exit, Result}; +use crate::host::sandbox::PreparedLaunch; +use crate::host::sandbox::distro::DISTRO_POLICY_REVISION; +use crate::host::sandbox::scratch::CLAIM_RELEASE_WAIT; +use crate::host::store::{LaunchId, LockedStore, OUTCOME_NAME, Recovery}; +use crate::host::supervisor::{self, Launched, VmmCommand}; +use crate::protocol::CAPABILITY_LEN; +use crate::{ErrorCode, ErrorKind}; + +/// Starts a provisioned sandbox, as `start` does. +pub(crate) fn start(state_path: &str, artifacts: &Artifacts<'_>, timeout: f64) -> Result { + let state_dir = prepare_state_directory(state_path, false)?; + let mut lock = LifecycleLock::acquire(&state_dir, timeout, false)?; + start_locked(&state_dir, lock.locked(), artifacts, timeout) +} + +/// Starts the sandbox of a locked state directory, as `_start` does, through the launch journal +/// of its store. +fn start_locked( + state_dir: &Path, + locked: &mut LockedStore, + artifacts: &Artifacts<'_>, + timeout: f64, +) -> Result { + let config_path = state_dir.join(CONFIG_NAME); + require_config(state_dir)?; + let config = read_config(&config_path)?; + // The release of a checkout without artifacts is downloaded before the runtime identity of a + // recorded launch is read from their install metadata, and no download replaces them until + // the start ends, once OpenVMM has read them, so that the runtime identity that the start + // records is theirs. A configuration without a hypervisor is malformed, which reading it below + // reports. + let _hold = match config.get("hypervisor").and_then(Value::as_str) { + Some(hypervisor) => Some(artifacts.ensure(hypervisor)?), + None => None, + }; + match find_vm(locked, artifacts)? { + Found::Nothing | Found::Exited => {} + Found::Live(_) => { + return Err(Exit::error( + ErrorCode::SandboxRunning, + "sandbox is already running or has stale runtime state", + )); + } + Found::Mismatched(vmm, differences) => return Err(mismatched(&vmm, &differences)), + } + let outcome_path = state_dir.join(OUTCOME_NAME); + unlink_missing_ok(&outcome_path)?; + let config = Config::new(config)?; + let launch = config.launch()?; + // The checks read no content of the distro layer, its manifest, or the scratch image, and + // never verify them again. The prepared launch holds the files that they return, and hands + // OpenVMM those objects: on Linux, OpenVMM inherits their descriptors; on Windows, it opens + // them by their final paths, which the files keep anything from replacing before OpenVMM, + // whose opens share them, has opened them. The scratch image's check also claims it for this + // sandbox, until OpenVMM exits. + let binding = config.distro_binding()?; + let scratch_binding = config.scratch_binding()?; + let distro = binding + .check(launch.distro(), DISTRO_POLICY_REVISION) + .map_err(binding_error)?; + // Reading the network checks its policy again, as provision did, before anything launches. + let launch = launch.with_network(config.network()?); + let scratch = scratch_binding + .check(launch.scratch(), Instant::now() + CLAIM_RELEASE_WAIT) + .map_err(binding_error)?; + // The plan takes the format and identity of the distro layer from its binding. + let plan = launch.bound_plan(&binding)?; + let [executable, kernel, initrd] = artifacts.require()?; + let runtime = artifacts.launch_identity()?; + let capability = random_bytes::()?; + if capability == [0; CAPABILITY_LEN] { + return Err(Exit::error( + ErrorCode::Other, + "the system returned an all-zero control capability", + )); + } + let memory_mib = config.memory_mib(); + let prepared = PreparedLaunch::new(plan, distro, Some(scratch), config.cmdline())?; + // The control endpoint, which the launch chooses, goes between these arguments. + let mut head = vec![executable]; + head.extend(prepared.openvmm_arguments().iter().cloned()); + head.extend( + [ + "--microvm-lifecycle", + "managed", + "--single-process", + "--hypervisor", + ] + .map(str::to_owned), + ); + head.push(config.hypervisor().to_owned()); + head.push("--memory".to_owned()); + head.push(format!("{memory_mib}M")); + head.extend(["--kernel".to_owned(), kernel, "--initrd".to_owned(), initrd]); + head.push("--cmdline".to_owned()); + head.push(prepared.kernel_command_line().to_owned()); + head.extend(["--virtio-console", "none", "--microvm-control-console"].map(str::to_owned)); + let tail = vec![ + "--microvm-control-auth-stdin".to_owned(), + "--microvm-report".to_owned(), + outcome_path.display().to_string(), + ]; + let directory = std::env::current_dir().map_err(Exit::from_os_error)?; + let mut command = Vec::new(); + let launch = LaunchId::generate().map_err(error_with_cause)?; + let mut starting = supervisor::launch(locked, &launch, &runtime, &capability, |endpoint| { + command = head; + command.push(format!("listen={endpoint}")); + command.extend(tail); + let arguments = command[1..].iter().map(OsString::from).collect(); + let mut vmm = VmmCommand::new(&command[0], arguments, directory); + // OpenVMM inherits the checked files that the prepared launch passes, and nothing else. + vmm.inherit = prepared.inherited(); + Ok(vmm) + }) + .map_err(error_with_cause)?; + let pid = starting.process().reference().pid; + let maps = prepared.plan().mapping_table(); + // Whether the guest agent acknowledged a request to shut down after a mapping failed. + let mut stopping = false; + let ready = + check_ready(starting.endpoint(), pid, &capability, timeout).and_then(|mut session| { + // Workloads run only once the guest agent has mapped every share in order, so a + // mapping that fails ends the start. A sandbox without shares maps nothing. + if maps.is_empty() { + return Ok(()); + } + session.map_host_paths(&maps, timeout).inspect_err(|error| { + // The agent then refuses workloads, so the guest shuts down as a stop has it do: + // it unmounts what it mapped, the export, the overlay, and the scratch disk before + // the VM powers off. A failed transfer closed the session, so a guest that stopped + // answering is terminated at once. + if !matches!(error, Exit::Interrupted) { + stopping = session.stop(timeout).is_ok(); + } + }) + }); + let Err(error) = ready else { + // The guest is ready, so OpenVMM holds the images: the checked files can go, which + // releases their guards, while OpenVMM keeps the scratch image's claim. + drop(prepared); + return starting.commit().map(drop).map_err(error_with_cause); + }; + // OpenVMM closes the control endpoint as it tears the VM down, before it publishes its + // outcome report and exits, so only a closed endpoint, or a guest that shuts down, warrants + // waiting for it. + let grace = if stopping || matches!(error, Exit::EndpointClosed) { + timeout + } else { + 0.0 + }; + let status = end_failed_start(starting.child(), grace, &command)?; + // OpenVMM has exited, so the launch ends: its capability, control socket, report staging + // file, and journal go, and its log and outcome report stay. + let unended = match starting.abandon() { + Ok(Recovery::Indeterminate(reason)) => Some(reason), + Ok(_) => None, + Err(failure) => Some(failure.message().to_owned()), + }; + // The checked files can go too, which releases their guards and the scratch image's claim. + drop(prepared); + let error = match status { + // A guest that shut down after a mapping failed exits as a stopped one does, so the + // mapping's failure tells why the start failed. + Some(status) if !stopping && !matches!(error, Exit::Interrupted) => Exit::error( + ErrorCode::Exited, + startup_exit_message(&outcome_path, status)?, + ), + _ => error, + }; + if let Some(reason) = unended + && let Some(message) = error.message() + { + return Err(Exit::error( + error.code().unwrap_or(ErrorCode::Other), + format!("{message}; the launch could not be ended, and remains for recovery: {reason}"), + )); + } + Err(error) +} + +/// Returns the error of a start whose distro layer or scratch image fails the check of its +/// binding: a binding that no longer holds needs a new provision, which verifies the layer and +/// binds the image again, while a scratch image that another sandbox claims, and a failure of the +/// host, do not. The error keeps its code. +fn binding_error(error: crate::Error) -> Exit { + match error.kind() { + ErrorKind::Io | ErrorKind::TimedOut => error.into(), + _ => Exit::error( + error.code(), + format!( + "{}; deprovision the sandbox and provision it again", + error.message() + ), + ), + } +} + +/// Waits until the guest of a VM that starts answers on `endpoint`, which OpenVMM process `pid` +/// serves, checks that it provides the features of [`REQUIRED_FEATURES`], and returns the session, +/// on which the caller sends the guest what it needs before the launch commits. +fn check_ready( + endpoint: &str, + pid: u32, + capability: &[u8; CAPABILITY_LEN], + timeout: f64, +) -> Result { + let mut session = Session::connect(endpoint, pid, capability, timeout)?; + session.ping(timeout)?; + let features = session.features(timeout)?; + let missing = features.missing(REQUIRED_FEATURES); + if !missing.is_empty() { + return Err(Exit::error( + ErrorCode::GuestFeaturesMissing, + format!( + "the guest image lacks control features that layered sandboxes need ({}); use \ + an NVX release or build that includes them", + missing.join(", ") + ), + )); + } + Ok(session) +} + +/// Ends the OpenVMM process of a failed start, as `_end_failed_start` does: returns OpenVMM's +/// status if it exits on its own within `grace` seconds, or terminates it and returns `None`. +/// Terminating OpenVMM while it publishes its outcome report would lose the report, and leave +/// only its staging file, which the store removes as the launch ends. +fn end_failed_start( + launched: &mut Launched, + grace: f64, + command: &[String], +) -> Result> { + let waited = if grace > 0.0 { + wait_interruptibly(launched, Deadline::after(grace)) + } else { + Ok(()) + }; + let status = launched + .wait_timeout(Duration::ZERO) + .map_err(Exit::from_os_error)?; + if status.is_none() { + launched.terminate().map_err(Exit::from_os_error)?; + let exited = launched + .wait_timeout(Duration::from_secs(5)) + .map_err(Exit::from_os_error)?; + if exited.is_none() { + launched.kill().map_err(Exit::from_os_error)?; + let killed = launched + .wait_timeout(Duration::from_secs(5)) + .map_err(Exit::from_os_error)?; + if killed.is_none() { + return Err(Exit::error( + ErrorCode::TimedOut, + format!("command {command:?} did not exit within 5 seconds of its kill"), + )); + } + } + } + // Ctrl-C during the grace period ends the start as interrupted, after OpenVMM. + waited?; + Ok(status) +} + +/// Waits until `launched` exits or `deadline` passes, failing if Ctrl-C arrives. +fn wait_interruptibly(launched: &mut Launched, deadline: Deadline) -> Result { + loop { + let slice = deadline + .remaining() + .map_or(RETRY_INTERVAL, |left| left.min(RETRY_INTERVAL)); + crate::host::cli::check_interrupt()?; + if launched + .wait_timeout(slice) + .map_err(Exit::from_os_error)? + .is_some() + || deadline.passed() + { + return Ok(()); + } + } +} + +/// Describes how OpenVMM ended a start, preferring its outcome report, which keeps the guest's +/// own status: its category and its integer status, when it holds both. +fn startup_exit_message(outcome_path: &Path, status: i64) -> Result { + let report = read_openvmm_outcome(outcome_path).ok(); + if let Some(outcome) = report.as_ref().and_then(|report| report.get("outcome")) + && let Some(Value::String(category)) = outcome.get("category") + && let Some(status_code) = outcome.get("status_code").and_then(Value::as_i64) + { + return Ok(format!( + "OpenVMM exited during startup: {category} status {status_code}" + )); + } + Ok(format!( + "OpenVMM exited during startup with status {status}" + )) +} + +/// Runs a workload in a running sandbox, as `exec_workload` does. The lock is held only while the +/// running VM is found, so a long workload cannot delay a stop. +pub(crate) fn exec_workload( + state_path: &str, + artifacts: &Artifacts<'_>, + request: &ExecRequest, + response_timeout: f64, +) -> Result { + let state_dir = prepare_state_directory(state_path, false)?; + let running = { + let mut lock = LifecycleLock::acquire(&state_dir, response_timeout, false)?; + running_vm(&state_dir, lock.locked(), artifacts)? + }; + let mut session = connect(&running.vmm, &running.capability, response_timeout)?; + session.exec(request, response_timeout) +} diff --git a/nvx-rs/src/host/cli/sandbox/lifecycle/stop.rs b/nvx-rs/src/host/cli/sandbox/lifecycle/stop.rs new file mode 100644 index 000000000..f4ae206eb --- /dev/null +++ b/nvx-rs/src/host/cli/sandbox/lifecycle/stop.rs @@ -0,0 +1,170 @@ +//! Stopping a managed sandbox and removing a stopped sandbox's state. + +use std::time::{Duration, Instant}; + +use super::directory::prepare_state_directory; +use super::outcome::read_openvmm_outcome; +use super::{Found, Running, connect, differing_parts, found_vm, running}; +use crate::ErrorCode; +use crate::host::cli::deadline::{Deadline, RETRY_INTERVAL}; +use crate::host::cli::sandbox::lock::LifecycleLock; +use crate::host::cli::sandbox::{Artifacts, error_with_cause}; +use crate::host::cli::{Exit, Result, check_interrupt, output}; +use crate::host::store::{IdentityPart, LockedStore, OUTCOME_NAME}; +use crate::host::supervisor::{StopOutcome, Vmm}; + +/// Stops a running sandbox, and checks the outcome report that its OpenVMM wrote, unless another +/// runtime identity than the artifacts' started the VM. +/// +/// The supervisor's stop runs one graceful step, which asks the guest to shut the VM down and +/// waits up to `timeout` seconds for OpenVMM to exit, so that Ctrl-C ends the wait. A VM whose +/// OpenVMM is still running then is reported, and keeps its runtime record: the stop never takes +/// the forced step. +/// +/// A VM of another runtime identity may speak other contracts, so the stop asks it nothing: it +/// ends the VM as [`terminate_mismatched`] does. +pub(crate) fn stop(state_path: &str, artifacts: &Artifacts<'_>, timeout: f64) -> Result { + let state_dir = prepare_state_directory(state_path, false)?; + let mut lock = LifecycleLock::acquire(&state_dir, timeout, false)?; + let locked = lock.locked(); + let Running { vmm, capability } = match found_vm(locked, artifacts)? { + Found::Mismatched(vmm, differences) => { + return terminate_mismatched(locked, &vmm, &differences, timeout); + } + found => running(&state_dir, locked, found)?, + }; + // The graceful step waits for the exit itself, so the stop's own wait, until a deadline that + // has passed by then, only checks whether OpenVMM has exited. + let stopped = vmm + .stop(locked, Instant::now(), None, |vmm, _| { + { + let mut session = connect(vmm, &capability, timeout)?; + session.stop(timeout)?; + } + wait_for_exit(vmm, Deadline::after(timeout)).map(drop) + }) + .map_err(error_with_cause)?; + if let Some(error) = stopped.graceful_error { + return Err(error); + } + if stopped.outcome == StopOutcome::TimedOut { + return Err(Exit::error( + ErrorCode::TimedOut, + "OpenVMM did not terminate after managed stop", + )); + } + read_openvmm_outcome(&state_dir.join(OUTCOME_NAME)).map(drop) +} + +/// Ends the VM `vmm`, which another runtime identity than the artifacts' started, and which +/// differs from them in `differences`, by identity-verified forced termination, and warns on +/// standard error that it did. +/// +/// Its guest is not asked to shut down, and no control session is opened to it: its OpenVMM is +/// terminated through its verified process identity, as [`Vmm::terminate`] does, and the stop +/// waits up to `timeout` seconds for it to exit, as [`terminate_within`] does, so that Ctrl-C ends +/// the wait. One that does not exit then, whose exit cannot be determined, or whose wait Ctrl-C +/// ends, keeps its runtime record, so that a later stop retries. +fn terminate_mismatched( + locked: &mut LockedStore, + vmm: &Vmm, + differences: &[IdentityPart], + timeout: f64, +) -> Result { + let pid = vmm.record().process().pid; + let parts = differing_parts(differences); + let outcome = terminate_within( + Deadline::after(timeout), + |bound| vmm.terminate(locked, bound).map_err(error_with_cause), + |deadline| wait_for_exit(vmm, deadline), + )?; + let ended = match outcome { + StopOutcome::Forced => "was terminated without a guest shutdown", + StopOutcome::Graceful => "had exited", + StopOutcome::TimedOut => { + return Err(Exit::error( + ErrorCode::TimedOut, + format!( + "OpenVMM process {pid} of the sandbox, which runs under another runtime \ + identity than these artifacts, did not exit after it was terminated" + ), + )); + } + }; + output::print_stderr(&format!( + "warning: sandbox ran under another runtime identity than these artifacts, which differ \ + in {parts}: its OpenVMM process {pid} {ended}" + )); + Ok(()) +} + +/// Ends an OpenVMM by identity-verified forced termination, and waits until `deadline` for it to +/// exit: `terminate` terminates it unless it has exited, waits up to the bound that it takes for +/// it to exit, and clears its runtime record once it has, as [`Vmm::terminate`] does, and `wait` +/// waits for its exit until a deadline, fails if Ctrl-C arrives first, and returns whether it +/// exited, as [`wait_for_exit`] does. +/// +/// The termination waits for one retry interval at most, and `wait` for the rest, so that Ctrl-C +/// ends the wait for an OpenVMM that does not exit at once, as one in uninterruptible I/O. Once +/// that one has exited, a second termination finds it exited, signals nothing, and clears its +/// record, and the result is [`StopOutcome::Forced`] all the same. One that still runs at the +/// deadline is [`StopOutcome::TimedOut`], and Ctrl-C fails the termination: both keep the record. +pub(super) fn terminate_within( + deadline: Deadline, + mut terminate: impl FnMut(Duration) -> Result, + wait: impl FnOnce(Deadline) -> Result, +) -> Result { + check_interrupt()?; + // A termination's bound is positive, though the deadline may have passed. + let bound = deadline + .remaining() + .map_or(RETRY_INTERVAL, |left| left.min(RETRY_INTERVAL)) + .max(Duration::from_nanos(1)); + match terminate(bound)? { + StopOutcome::TimedOut if wait(deadline)? => match terminate(bound)? { + StopOutcome::TimedOut => Ok(StopOutcome::TimedOut), + StopOutcome::Graceful | StopOutcome::Forced => Ok(StopOutcome::Forced), + }, + outcome => Ok(outcome), + } +} + +/// Waits until the OpenVMM of `vmm` exits or `deadline` passes, and returns whether it exited, +/// failing if Ctrl-C arrives first. +fn wait_for_exit(vmm: &Vmm, deadline: Deadline) -> Result { + loop { + check_interrupt()?; + let slice = deadline + .remaining() + .map_or(RETRY_INTERVAL, |left| left.min(RETRY_INTERVAL)); + if vmm + .wait(Instant::now() + slice) + .map_err(Exit::from_os_error)? + { + return Ok(true); + } + if deadline.passed() { + return Ok(false); + } + } +} + +/// Removes a stopped sandbox and its state directory, as `deprovision` does. +pub(crate) fn deprovision(state_path: &str, artifacts: &Artifacts<'_>, timeout: f64) -> Result { + let state_dir = prepare_state_directory(state_path, false)?; + let mut lock = LifecycleLock::acquire(&state_dir, timeout, false)?; + match found_vm(lock.locked(), artifacts)? { + Found::Nothing | Found::Exited => {} + Found::Live(_) | Found::Mismatched(..) => { + return Err(Exit::error( + ErrorCode::SandboxRunning, + "sandbox must be stopped before deprovision", + )); + } + } + // No OpenVMM of the sandbox runs, and a start that was interrupted is over, so the store + // removes the sandbox: its files and the configuration, then refuses files that are not + // NVX's, and otherwise removes its lock file, while it still holds the lock, and the + // directory. + lock.remove(timeout) +} diff --git a/nvx-rs/src/host/cli/sandbox/lifecycle/tests.rs b/nvx-rs/src/host/cli/sandbox/lifecycle/tests.rs new file mode 100644 index 000000000..85b6ebbcd --- /dev/null +++ b/nvx-rs/src/host/cli/sandbox/lifecycle/tests.rs @@ -0,0 +1,118 @@ +use std::cell::{Cell, RefCell}; + +use super::stop::terminate_within; +use super::*; +use crate::host::cli::deadline::Deadline; +use crate::host::supervisor::StopOutcome; + +/// An OpenVMM of another runtime identity whose termination a test scripts, without a process. +struct Scripted { + /// Whether OpenVMM has exited. + exited: Cell, + /// Whether a termination ends OpenVMM within its bound, which one in uninterruptible I/O + /// outlives. + dies_when_terminated: bool, + /// Whether the runtime record of its launch remains. + recorded: Cell, + /// The bounds of its terminations, in order. + bounds: RefCell>, +} + +impl Scripted { + /// Returns an OpenVMM that runs, which a termination ends if `dies_when_terminated`. + fn running(dies_when_terminated: bool) -> Self { + Self { + exited: Cell::new(false), + dies_when_terminated, + recorded: Cell::new(true), + bounds: RefCell::new(Vec::new()), + } + } + + /// Terminates OpenVMM within `bound`, as [`Vmm::terminate`] does: one that has exited is + /// not signaled, and the runtime record goes once it has exited. + fn terminate(&self, bound: Duration) -> Result { + self.bounds.borrow_mut().push(bound); + let outcome = if self.exited.get() { + StopOutcome::Graceful + } else if self.dies_when_terminated { + self.exited.set(true); + StopOutcome::Forced + } else { + StopOutcome::TimedOut + }; + if self.exited.get() { + self.recorded.set(false); + } + Ok(outcome) + } + + /// Ends OpenVMM, as a stop with an hour's timeout does, and ends the stop's wait for its + /// exit with `waited`: OpenVMM exits during the wait if that is `Ok(true)`. + fn stop(&self, waited: Result) -> Result { + terminate_within( + Deadline::after(3600.0), + |bound| self.terminate(bound), + |_| { + if matches!(waited, Ok(true)) { + self.exited.set(true); + } + waited + }, + ) + } + + /// Returns the bounds of its terminations. + fn bounds(&self) -> Vec { + self.bounds.borrow().clone() + } +} + +#[test] +fn ctrl_c_ends_the_wait_for_a_terminated_openvmm_that_has_not_exited() { + // OpenVMM outlives its termination, so the termination waits one retry interval at most, + // whatever the timeout, and the stop's own wait, which Ctrl-C ends, waits for the rest. + let openvmm = Scripted::running(false); + let stopped = openvmm.stop(Err(Exit::Interrupted)); + assert!(matches!(stopped, Err(Exit::Interrupted)), "{stopped:?}"); + let bounds = openvmm.bounds(); + assert_eq!(bounds.len(), 1, "{bounds:?}"); + assert!(bounds[0] <= RETRY_INTERVAL, "{bounds:?}"); + // Its exit was not seen, so its runtime record remains for the next stop. + assert!(openvmm.recorded.get()); +} + +#[test] +fn a_terminated_openvmm_that_exits_during_the_wait_was_forced() { + // Once it has exited, a second termination finds it exited, signals nothing, and clears + // its runtime record. + let openvmm = Scripted::running(false); + let stopped = openvmm.stop(Ok(true)); + assert!(matches!(stopped, Ok(StopOutcome::Forced)), "{stopped:?}"); + assert_eq!(openvmm.bounds().len(), 2); + assert!(!openvmm.recorded.get()); + + // One that still runs at the deadline keeps its record. + let survivor = Scripted::running(false); + let stopped = survivor.stop(Ok(false)); + assert!(matches!(stopped, Ok(StopOutcome::TimedOut)), "{stopped:?}"); + assert_eq!(survivor.bounds().len(), 1); + assert!(survivor.recorded.get()); +} + +#[test] +fn an_openvmm_that_its_termination_ends_or_that_exited_is_not_waited_for() { + // The stop's wait, which would fail here, does not run. + let openvmm = Scripted::running(true); + let stopped = openvmm.stop(Err(Exit::Interrupted)); + assert!(matches!(stopped, Ok(StopOutcome::Forced)), "{stopped:?}"); + assert_eq!(openvmm.bounds().len(), 1); + assert!(!openvmm.recorded.get()); + + let exited = Scripted::running(true); + exited.exited.set(true); + let stopped = exited.stop(Err(Exit::Interrupted)); + assert!(matches!(stopped, Ok(StopOutcome::Graceful)), "{stopped:?}"); + assert_eq!(exited.bounds().len(), 1); + assert!(!exited.recorded.get()); +} diff --git a/nvx-rs/src/host/filesystem.rs b/nvx-rs/src/host/filesystem.rs deleted file mode 100644 index 20e8a6b0c..000000000 --- a/nvx-rs/src/host/filesystem.rs +++ /dev/null @@ -1,3734 +0,0 @@ -//! Host paths exposed to the guest through OpenVMM's aggregate virtio-fs export, as the one -//! planner of every NVX caller plans them, for the blockless root and for the container root of a -//! layered sandbox alike. -//! -//! OpenVMM offers one virtio-fs device. [`plan`] attaches it as an aggregate: a synthetic, -//! read-only root that lists one child per exported host directory or file, named by its index, -//! which the guest mounts at a directory that only the guest's root can enter. The guest then -//! bind-mounts each mapped path at its target, read-only or read-write, in the order of the plan's -//! [`Bind`]s: from the mapping table that it receives over the control channel, or from boot -//! tokens that a caller renders from the same binds. -//! -//! An [`ExportRequest`] names the mapped host paths, each read-only or read-write at its guest -//! target; the host paths that OpenVMM hides inside them, and the paths inside those that stay -//! visible; the paths to which the writes of a read-write directory are narrowed; the host -//! identity of the file operations; and the guest targets that the guest's root keeps for itself. -//! -//! - A caller gives each target, or lets MXC's convention for Linux guests derive it from the -//! resolved host path: a Windows path `C:\work\src` appears at `/mnt/c/work/src`, and a Linux -//! path at the same path ([`guest_path`]). -//! - The reserved targets depend on the root, and the caller supplies them ([`ReservedTargets`]): -//! the blockless root keeps the guest's system trees, and the container root of a layered -//! sandbox keeps the trees of its runtime. -//! -//! The children are the outermost mapped directories and the mapped files outside them. OpenVMM -//! exports each such file alone, without its directory, so mapped paths may lie anywhere, on any -//! volume, except that a whole volume cannot be mapped. A child is read-write only if it holds a -//! read-write mapping, and OpenVMM then limits writes to the child's writable parts, so the host -//! enforces each mapping's access itself, and guest mount flags are never the boundary. OpenVMM -//! cannot keep a path read-only inside a writable part, so a read-only path there is refused on -//! every host, while a read-write path inside a read-only directory is not. OpenVMM also hides the -//! denied paths, except for the allowed paths inside them. -//! -//! The plan records the identities of the mapped and denied paths that a workload could move, -//! those inside a writable part, and [`ExportPlan::verify`] checks them again before each start. -//! A policy that would need more children than OpenVMM offers is refused rather than served -//! through the directories of its files, which would grant more than the files. - -use std::fs; -use std::path::{Component, Path, PathBuf, Prefix}; - -use serde::{Deserialize, Serialize}; - -use super::platform; -use crate::error::{Error, Result}; -use crate::protocol::export::EXPORT_DIRECTORY; -use crate::protocol::share::{RESERVED_EXACT_MOUNT_TARGETS, RESERVED_MOUNT_TARGETS}; -use crate::protocol::{HostMap, MAPS_TOKEN, MAX_HOST_MAPPINGS}; - -/// Guest directory where the guest's init mounts the export, which the export's `virtfs_dir=` -/// boot token names, and guests check: [`EXPORT_DIRECTORY`]. Only the guest's root can enter its -/// parent. -pub const GUEST_EXPORT: &str = EXPORT_DIRECTORY; - -/// Largest number of host directories and files that OpenVMM exports through one aggregate. -pub const MAX_CHILDREN: usize = 256; -/// Largest number of hidden, allowed, or writable paths that OpenVMM accepts in one exported -/// directory. -const MAX_POLICY_PATHS: usize = 128; -/// Largest combined UTF-8 byte length of one exported directory's paths of one kind. -const MAX_POLICY_BYTES: usize = 16 * 1024; -/// Largest length of one such path. -const MAX_POLICY_PATH_BYTES: usize = 4096; -/// Largest combined UTF-8 byte length of all policies in one aggregate export. -const MAX_AGGREGATE_POLICY_BYTES: usize = 128 * 1024; -/// Longest guest path, without the terminating NUL. -const MAX_GUEST_PATH: usize = 4095; - -/// Guest directories that a mapping must not cover in the blockless root, because the guest's own -/// system lives there. -pub const RESERVED_TARGETS: [&str; 13] = [ - "/bin", "/boot", "/dev", "/etc", "/lib", "/proc", "/root", "/run", "/sbin", "/sys", "/usr", - "/var", "/init", -]; - -/// Why a request for the caller owner fails on a Windows host. -const CALLER_OWNER_ON_WINDOWS: &str = "the caller owner requires a Linux host; Windows has no \ - per-request POSIX identity for OpenVMM to switch to"; - -/// The guest targets that a root keeps for its own system or runtime, which no mapping may take. -/// -/// A mapping may be neither at nor below one of the [`trees`](Self::trees), nor at one of their -/// ancestors, where its bind would shadow the tree, nor at one of the [`exact`](Self::exact) -/// paths, though it may be below them. No mapping is ever at the guest's root, `/`. -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -pub struct ReservedTargets<'a> { - /// Guest trees that the root owns. - pub trees: &'a [&'a str], - /// Guest paths at which the root mounts or creates something, below which mappings may lie. - pub exact: &'a [&'a str], -} - -impl ReservedTargets<'static> { - /// The blockless root, in which the guest's own system runs: [`RESERVED_TARGETS`]. - pub const BLOCKLESS_ROOT: Self = Self { - trees: &RESERVED_TARGETS, - exact: &[], - }; - - /// The container root of a layered sandbox: the trees in which its runtime mounts, binds, or - /// stages something, [`RESERVED_MOUNT_TARGETS`], and exactly the directories on the way to - /// them that it creates, [`RESERVED_EXACT_MOUNT_TARGETS`], which the guest checks as well. - pub const CONTAINER_ROOT: Self = Self { - trees: &RESERVED_MOUNT_TARGETS, - exact: &RESERVED_EXACT_MOUNT_TARGETS, - }; -} - -impl<'a> ReservedTargets<'a> { - /// Returns the reserved path that a mapping at `target`, an absolute guest path, would take - /// or shadow, or `None` if `target` is free. - pub fn reservation(&self, target: &str) -> Option<&'a str> { - if target == "/" { - return Some("/"); - } - self.exact - .iter() - .copied() - .find(|exact| *exact == target) - .or_else(|| { - self.trees - .iter() - .copied() - .find(|tree| within(target, tree) || within(tree, target)) - }) - } -} - -/// The host identity under which OpenVMM runs the file operations of an export. -#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Hash, Serialize, Deserialize)] -#[serde(rename_all = "lowercase")] -pub enum Owner { - /// Every operation runs as OpenVMM, the default. - #[default] - Vmm, - /// Each operation runs as the identity of the guest caller, with guest root squashed to the - /// owner of the exported directory or file. Linux hosts only. - Caller, -} - -impl Owner { - /// Returns whether this is the default owner, which the serde form of a plan leaves out. - fn is_vmm(&self) -> bool { - *self == Self::Vmm - } -} - -/// The host paths of one sandbox, as [`plan`] exports them: the children of OpenVMM's aggregate -/// export, the bind mounts of the guest, and the host identity of the file operations. -/// -/// Callers record the plan, so its serde form is part of their state: a caller versions its -/// records, and a record of another version is not read. -#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] -#[serde(rename_all = "camelCase", deny_unknown_fields)] -pub struct ExportPlan { - /// Children of the aggregate export; each is named by its index. - pub children: Vec, - /// Bind mounts the guest creates, parents before children. - pub binds: Vec, - /// The host identity of the file operations; a plan of the default owner leaves it out. - #[serde(default, skip_serializing_if = "Owner::is_vmm")] - pub owner: Owner, -} - -/// One host directory or file of the aggregate export. -#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] -#[serde(rename_all = "camelCase", deny_unknown_fields)] -pub struct ExportChild { - /// Canonical host directory, or the canonical host file that the child exposes alone. - pub root: PathBuf, - /// Whether the root is a regular file, which OpenVMM exposes without its directory; a child - /// of a directory leaves it out. - #[serde(default, skip_serializing_if = "std::ops::Not::not")] - pub file: bool, - /// Whether the guest may modify it, which a read-write mapping inside it requires. - pub writable: bool, - /// Canonical host paths inside the child that OpenVMM hides; the root itself, when it shows - /// only its allowed paths. - pub denied: Vec, - /// Identities, at provision, of the denied paths that a workload could move: those inside a - /// writable part. - pub denied_identities: Vec>, - /// Canonical paths inside denied paths that stay visible. - pub allowed: Vec, - /// Canonical paths to which OpenVMM limits the guest's writes; empty if the guest may modify - /// everything that it can reach in a writable child. - pub write: Vec, -} - -/// Device and file index of a host object, which survive renames. -#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] -#[serde(rename_all = "camelCase", deny_unknown_fields)] -pub struct FileIdentity { - /// Device, or the volume serial number on Windows. - pub device: u64, - /// Inode, or the file index on Windows. - pub index: u64, -} - -/// One mapped host path. -#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] -#[serde(rename_all = "camelCase", deny_unknown_fields)] -pub struct Bind { - /// Index of the child that holds the mapped path. - pub child: usize, - /// Path relative to the child's root, `/`-separated; empty for the root itself. - pub source: String, - /// Absolute guest path. - pub target: String, - /// Whether the guest mounts it read-only. - pub read_only: bool, - /// Identity of the host object at provision, when it lies inside a writable part and a - /// workload could therefore move it. - #[serde(default, skip_serializing_if = "Option::is_none")] - pub identity: Option, -} - -impl Bind { - /// Path of the mapped object relative to the guest's mount of the export. - pub fn guest_source(&self) -> String { - guest_source(self.child, &self.source) - } -} - -/// Joins an aggregate child index and child-relative source into a guest export path. -fn guest_source(child: usize, source: &str) -> String { - if source.is_empty() { - child.to_string() - } else { - format!("{child}/{source}") - } -} - -/// Returns the guest path at which a sandbox exposes the host path `path`, or `None` if it -/// has no guest equivalent (relative paths, UNC paths, or paths that are not valid UTF-8). -/// -/// A Windows path `C:\work\src` maps to `/mnt/c/work/src`, and an absolute Linux path maps to -/// itself. `.` components are dropped; `..` components are rejected. -pub fn guest_path(path: &Path) -> Option { - let mut components = path.components().peekable(); - let mut guest = match components.peek()? { - Component::Prefix(prefix) => { - let drive = match prefix.kind() { - Prefix::Disk(drive) | Prefix::VerbatimDisk(drive) => drive, - _ => return None, - }; - components.next(); - if !matches!(components.next()?, Component::RootDir) { - return None; - } - format!("/mnt/{}", char::from(drive).to_ascii_lowercase()) - } - Component::RootDir if !cfg!(windows) => { - components.next(); - String::new() - } - _ => return None, - }; - for component in components { - match component { - Component::Normal(name) => { - guest.push('/'); - guest.push_str(name.to_str()?); - } - Component::CurDir => {} - _ => return None, - } - } - if guest.is_empty() { - guest.push('/'); - } - Some(guest) -} - -/// Returns the guest path of an existing host path the way mappings derive it: from the -/// resolved path, so links, letter case, and short names do not matter. A path that does not -/// exist is translated as written. -/// -/// Use it to turn a host working directory into a `process.cwd` inside a mapped path. -pub fn resolve_guest_path(path: &Path) -> Option { - match canonicalize(path) { - Ok(canonical) => guest_path(&canonical), - Err(_) => guest_path(path), - } -} - -/// Where the guest sees a mapped host path. -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -pub enum Target<'a> { - /// At the guest path that [`guest_path`] derives from the resolved host path. - Convention, - /// At this absolute guest path: a path other than the root, of at most 4,095 bytes, without - /// NUL characters or empty, `.`, or `..` components. - Guest(&'a str), -} - -/// A host directory or regular file that the guest reads, or reads and writes, at a target. -#[derive(Debug, Clone, Copy)] -pub struct Mapping<'a> { - /// How messages name the list or option that requested the mapping, such as - /// `filesystem.readonlyPaths`. - pub name: &'a str, - /// The host path. - pub path: &'a Path, - /// Where the guest sees it. - pub target: Target<'a>, - /// Whether the guest may only read it. - pub read_only: bool, -} - -impl<'a> Mapping<'a> { - /// Returns the mappings of the paths of `list`, under its name, at the guest paths of the - /// convention, read-only or read-write. - pub fn convention(list: PathList<'a>, read_only: bool) -> impl Iterator + 'a { - list.paths.iter().map(move |path| Self { - name: list.name, - path, - target: Target::Convention, - read_only, - }) - } -} - -/// A list of host paths, and the name by which messages refer to its entries, such as -/// `readOnly`. -#[derive(Debug, Clone, Copy)] -pub struct PathList<'a> { - /// The name of the list in messages. - pub name: &'a str, - /// The host paths. - pub paths: &'a [PathBuf], -} - -impl PathList<'static> { - /// A list without paths. - pub const EMPTY: Self = Self { - name: "", - paths: &[], - }; -} - -/// The host paths that a sandbox maps, those that it hides inside them and shows again inside -/// those, the paths to which it narrows writes, the host identity of the file operations, and the -/// guest targets that the guest's root keeps for itself. -#[derive(Debug, Clone, Copy)] -pub struct ExportRequest<'a> { - /// The mapped host paths, each at its target with its access. - pub mappings: &'a [Mapping<'a>], - /// Host paths hidden inside the mapped paths. A mapped directory may be denied itself when - /// allowed paths inside it stay visible. - pub denied: PathList<'a>, - /// Host paths inside denied paths that stay visible, with the directories on the way to - /// them listing only them. - pub allowed: PathList<'a>, - /// Host paths to which the guest's writes are narrowed inside the read-write directory that - /// they lie in, which must be mapped whole: the guest may then write only them and the - /// read-write mappings inside that directory. - pub write: PathList<'a>, - /// The host identity of the file operations. - pub owner: Owner, - /// The guest targets that the guest's root keeps for itself. - pub reserved: ReservedTargets<'a>, -} - -impl<'a> ExportRequest<'a> { - /// Returns the request that maps `mappings` into a root that reserves `reserved`, without - /// denied, allowed, or write paths, as the VMM. - pub fn new(mappings: &'a [Mapping<'a>], reserved: ReservedTargets<'a>) -> Self { - Self { - mappings, - denied: PathList::EMPTY, - allowed: PathList::EMPTY, - write: PathList::EMPTY, - owner: Owner::Vmm, - reserved, - } - } -} - -/// Plans the export of `request`, or returns `None` if it maps nothing. -/// -/// Mapped paths must exist, and resolve to paths that are valid UTF-8. A mapping's target is the -/// one that the request gives, or the guest path that the convention derives from the resolved -/// host path, so two spellings of one object (links, letter case, or short names) map to one guest -/// path; either must be free in the request's [`ReservedTargets`]. An object mapped twice at one -/// target is mapped once, read-only if either mapping is. Two objects never share a target, and a -/// target inside another mapping's must show the very object that the other mapping shows there. -/// A whole volume cannot be mapped, and a policy that would need more children than OpenVMM -/// offers is refused. -/// -/// A read-only path must not lie inside a writable part, where the host could not keep it -/// read-only: inside a read-write directory, unless write paths narrow that directory's writes -/// to other paths, or inside a write path. A read-write path may lie inside a read-only -/// directory. A write path must lie inside a read-write directory that a child exports whole. -/// -/// A denied path must not hide a mapped path, except that a denied mapped directory shows the -/// allowed paths inside it, unless it is a read-write mapping inside a directory that is not -/// writable as a whole: OpenVMM narrows that directory's writes to the mapping, and it cannot make -/// a hidden path writable. A denied path that does not exist yet must not lie inside a mapped -/// path, because nothing could hide it once a workload creates it; denied paths outside the -/// mapped paths are ignored. An allowed path must lie inside a denied path, with no other allowed -/// path between them, and a write path must not be hidden. The caller owner needs a Linux host, -/// on which no exported directory or file may belong to UID 0 or GID 0. -/// -/// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) when the request breaks -/// these rules or exceeds a limit of OpenVMM or the guest agent. -pub fn plan(request: &ExportRequest<'_>) -> Result> { - let unsupported = |message: String| Err(Error::unsupported(message)); - if request.owner == Owner::Caller && cfg!(windows) { - return unsupported(CALLER_OWNER_ON_WINDOWS.to_owned()); - } - let mut mapped: Vec> = Vec::new(); - for mapping in request.mappings { - let entry = resolve_mapping(mapping, request.reserved)?; - match mapped - .iter_mut() - .find(|other| other.canonical == entry.canonical && other.target == entry.target) - { - // The same object listed twice at one target keeps its most restrictive access, and - // the name and spelling of the entry that gave it. - Some(other) => { - if entry.read_only && !other.read_only { - *other = entry; - } - } - None => mapped.push(entry), - } - } - if mapped.is_empty() { - return Ok(None); - } - if mapped.len() > MAX_HOST_MAPPINGS { - return unsupported(format!( - "the openvmm backend maps at most {MAX_HOST_MAPPINGS} paths into one sandbox" - )); - } - check_targets(&mapped)?; - let roots = outermost(mapped.iter().map(|entry| entry.canonical.clone()).collect()); - let write = write_paths(request.write, &mapped, &roots)?; - check_read_only(&mapped, &write, request.write.name, &roots)?; - distinct_roots(&roots)?; - if roots.len() > MAX_CHILDREN { - // Exporting directories for some of the files would grant more than the files. - return unsupported(format!( - "the mapped paths need {} exported directories and files, but OpenVMM exports at \ - most {MAX_CHILDREN}; map directories that contain the files instead", - roots.len() - )); - } - let allowed = allowed_paths(request.allowed, &roots)?; - let denied = denied_paths(request.denied, &mapped, &allowed)?; - check_allowed_paths(request.allowed.name, request.denied.name, &allowed, &denied)?; - check_write_paths( - request.write.name, - request.denied.name, - &write, - &denied, - &allowed, - )?; - let denied = canonicals(&denied); - let mut allowed = canonicals(&allowed); - allowed.sort(); - - let parts: Vec = roots - .iter() - .map(|root| writable_part(root, &mapped, &write)) - .collect(); - // A read-write mapping inside a child that is not writable as a whole becomes one of the - // child's write paths, and OpenVMM refuses to hide a write path. The write paths of the - // request were checked already. - for (root, part) in roots.iter().zip(&parts) { - for path in &part.write { - if let Some((rule, true)) = nearest_rule(path, &denied, &allowed, true) { - let (name, given) = mapped - .iter() - .find(|entry| entry.canonical == *path) - .map_or((request.write.name, path.as_path()), |entry| { - (entry.name, entry.given.as_path()) - }); - return unsupported(format!( - "{} entry {} hides the {name} entry {}, to which OpenVMM narrows the writes \ - of {}, and OpenVMM cannot make a hidden path writable; map read-write the \ - paths that it shows again instead", - request.denied.name, - rule.display(), - given.display(), - root.display() - )); - } - } - } - // Only objects inside a writable part can be swapped by a workload; pinning others would - // make ordinary host edits, which often replace files, block the next start. - let movable = |path: &Path| { - roots - .iter() - .zip(&parts) - .any(|(root, part)| path.starts_with(root) && part.moves(root, path)) - }; - let mut children = Vec::with_capacity(roots.len()); - let mut policy_bytes = 0; - for (root, part) in roots.iter().zip(&parts) { - // Every root is a mapping itself, so the guest reaches all of it. - let Some(own) = mapped.iter().find(|entry| entry.canonical == *root) else { - return Err(Error::other(format!( - "{} is exported but not mapped", - root.display() - ))); - }; - let inside = |paths: &[PathBuf]| -> Vec { - paths - .iter() - .filter(|path| path.starts_with(root)) - .cloned() - .collect() - }; - let child_allowed = inside(&allowed); - let child_denied = needed_denied_paths(&inside(&denied), &child_allowed); - check_root(root)?; - policy_bytes += check_policy_paths(root, &child_denied, "hide")?; - policy_bytes += check_policy_paths(root, &child_allowed, "show again")?; - policy_bytes += check_policy_paths(root, &part.write, "make writable")?; - let denied_identities = child_denied - .iter() - .map(|path| movable(path).then(|| identity(path)).transpose()) - .collect::>>()?; - children.push(ExportChild { - root: root.clone(), - file: !own.directory, - writable: part.writable, - denied: child_denied, - denied_identities, - allowed: child_allowed, - write: part.write.clone(), - }); - } - if policy_bytes > MAX_AGGREGATE_POLICY_BYTES { - return unsupported(format!( - "the paths that OpenVMM is to hide, show again, or make writable exceed its \ - {MAX_AGGREGATE_POLICY_BYTES}-byte limit" - )); - } - - let mut binds = Vec::with_capacity(mapped.len()); - for entry in &mapped { - let Some(child) = children - .iter() - .position(|child| entry.canonical.starts_with(&child.root)) - else { - return Err(Error::other(format!( - "{} lies in no exported directory", - entry.given.display() - ))); - }; - let Some(source) = relative_text(&children[child].root, &entry.canonical) else { - return unsupported(format!("{} is not valid UTF-8", entry.given.display())); - }; - if GUEST_EXPORT.len() + 1 + guest_source(child, &source).len() > MAX_GUEST_PATH - || entry.target.len() > MAX_GUEST_PATH - { - return unsupported(format!( - "{} is too long for a guest path", - entry.given.display() - )); - } - binds.push(Bind { - child, - source, - target: entry.target.clone(), - read_only: entry.read_only, - identity: movable(&entry.canonical) - .then(|| identity(&entry.canonical)) - .transpose()?, - }); - } - binds.sort_by(|left, right| { - depth(&left.target) - .cmp(&depth(&right.target)) - .then_with(|| left.target.cmp(&right.target)) - }); - if request.owner == Owner::Caller { - check_caller_roots(&children)?; - } - let plan = ExportPlan { - children, - binds, - owner: request.owner, - }; - // Callers read a record of the plan back through `check`, which accepts every plan that this - // function makes. - debug_assert!( - plan.check(request.reserved).is_ok(), - "the planner made a plan that its check refuses: {:?}", - plan.check(request.reserved) - ); - Ok(Some(plan)) -} - -/// Resolves one requested mapping: its host path must exist as a directory or regular file that -/// is not a whole volume, and its target must be a guest path that `reserved` leaves free. -fn resolve_mapping<'a>(mapping: &Mapping<'a>, reserved: ReservedTargets<'_>) -> Result> { - let unsupported = |message: String| Err(Error::unsupported(message)); - let Mapping { - name, - path, - target, - read_only, - } = *mapping; - let canonical = canonicalize(path).map_err(|error| { - Error::unsupported(format!("{name} entry {} is not accessible", path.display())) - .with_source(error) - })?; - // OpenVMM's options and the records of a plan carry host paths as text. - if canonical.to_str().is_none() { - return unsupported(format!( - "{name} entry {} resolves to a path that is not valid UTF-8", - path.display() - )); - } - let metadata = fs::metadata(&canonical).map_err(|error| { - Error::unsupported(format!("cannot inspect {}", path.display())).with_source(error) - })?; - if !metadata.is_dir() && !metadata.is_file() { - return unsupported(format!( - "{name} entry {} is neither a directory nor a regular file", - path.display() - )); - } - if metadata.is_dir() - && let Some(reason) = volume_root_refusal(&canonical) - { - return unsupported(format!("{name} entry {} {reason}", path.display())); - } - let target = match target { - Target::Convention => { - let Some(target) = guest_path(&canonical) else { - return unsupported(format!( - "{name} entry {} has no guest path; use an absolute local path", - path.display() - )); - }; - if let Some(reserved) = reserved.reservation(&target) { - return unsupported(format!( - "{name} entry {} would cover the guest's {reserved}", - path.display() - )); - } - target - } - Target::Guest(target) => { - if let Some(problem) = target_problem(target) { - return unsupported(format!( - "{name} entry {} has the guest target {target:?}, which {problem}", - path.display() - )); - } - if let Some(reserved) = reserved.reservation(target) { - return unsupported(format!( - "{name} entry {} has the guest target {target}, which would cover the \ - guest's {reserved}", - path.display() - )); - } - target.to_owned() - } - }; - Ok(Mapped { - name, - given: path.to_path_buf(), - canonical, - target, - read_only, - directory: metadata.is_dir(), - }) -} - -/// Explains why `target` cannot be a guest target, or returns `None` if it can: it must be an -/// absolute path other than the root, of at most [`MAX_GUEST_PATH`] bytes, without NUL characters -/// or empty, `.`, or `..` components. -fn target_problem(target: &str) -> Option<&'static str> { - let Some(rest) = target.strip_prefix('/') else { - return Some("is not absolute"); - }; - if rest.is_empty() { - return Some("is the guest's root"); - } - if target.len() > MAX_GUEST_PATH { - return Some("is longer than a guest path may be"); - } - if target.contains('\0') { - return Some("contains a NUL character"); - } - if rest - .split('/') - .any(|component| matches!(component, "" | "." | "..")) - { - return Some("has an empty, `.`, or `..` component"); - } - None -} - -/// Refuses mappings whose targets collide: two objects at one target, or a target inside another -/// mapping's that does not show the very object that the other mapping shows there, which it -/// would hide. -fn check_targets(mapped: &[Mapped<'_>]) -> Result<()> { - for (index, entry) in mapped.iter().enumerate() { - for other in &mapped[..index] { - // The entries of one object at one target are merged already. - if entry.target == other.target { - return Err(Error::unsupported(format!( - "{} entry {} and the {} entry {} have the same guest target {}", - entry.name, - entry.given.display(), - other.name, - other.given.display(), - entry.target - ))); - } - let (outer, inner) = if within(&entry.target, &other.target) { - (other, entry) - } else if within(&other.target, &entry.target) { - (entry, other) - } else { - continue; - }; - if !shows(outer, inner) { - return Err(Error::unsupported(format!( - "{} entry {} at {} would hide part of the {} entry {} at {}; map it at a \ - target outside that one", - inner.name, - inner.given.display(), - inner.target, - outer.name, - outer.given.display(), - outer.target - ))); - } - } - } - Ok(()) -} - -/// Returns whether `inner`, whose target lies below that of `outer`, maps the host object that -/// `outer` shows at that target: whether the rest of `inner`'s target spells, name for name, the -/// host path from `outer`'s object to `inner`'s, as a plan's bind sources spell it. A Windows host -/// reads a backslash in a name as a separator, and a name with a root or a drive as another path, -/// so such a guest name never matches. -fn shows(outer: &Mapped<'_>, inner: &Mapped<'_>) -> bool { - let Some(rest) = inner - .target - .strip_prefix(outer.target.as_str()) - .and_then(|rest| rest.strip_prefix('/')) - else { - return false; - }; - outer.directory - && relative_text(&outer.canonical, &inner.canonical).is_some_and(|names| names == rest) -} - -/// A path of an access policy, resolved. -struct PolicyPath { - /// Host path as the caller supplied it, for diagnostics. - given: PathBuf, - /// Link-resolved host path. - canonical: PathBuf, -} - -/// Returns the resolved paths of `paths`. -fn canonicals(paths: &[PolicyPath]) -> Vec { - paths.iter().map(|path| path.canonical.clone()).collect() -} - -/// Resolves the path of the list `field`, which must exist. -fn resolve_policy_path(field: &str, path: &Path) -> Result { - canonicalize(path).map_err(|error| { - Error::unsupported(format!( - "{field} entry {} is not accessible", - path.display() - )) - .with_source(error) - }) -} - -/// Resolves the write paths of `list`: each must lie inside a read-write directory that a child -/// of `roots` exports whole, other than that directory itself. -fn write_paths( - list: PathList<'_>, - mapped: &[Mapped<'_>], - roots: &[PathBuf], -) -> Result> { - let field = list.name; - let mut paths = Vec::with_capacity(list.paths.len()); - for path in list.paths { - let canonical = resolve_policy_path(field, path)?; - let narrows = roots - .iter() - .find(|root| canonical.starts_with(root)) - .is_some_and(|root| { - *root != canonical - && mapped.iter().any(|entry| { - entry.canonical == *root && entry.directory && !entry.read_only - }) - }); - if !narrows { - return Err(Error::unsupported(format!( - "{field} entry {} lies in no directory that is mapped read-write as a whole, \ - whose writes it could narrow", - path.display() - ))); - } - paths.push(PolicyPath { - given: path.clone(), - canonical, - }); - } - Ok(paths) -} - -/// Refuses a read-only mapping that the host could not keep read-only: one inside a write path, -/// one that a read-write mapping holds, unless write paths narrow that mapping's writes, or one of -/// an object that a read-write mapping maps elsewhere. OpenVMM can limit a child's writes to some -/// of its paths, but cannot keep a path inside them read-only. Only the guest's bind mount would -/// then protect the path, and guest root can bypass it: through the export, or on a Windows host, -/// which opens files case-insensitively, through another spelling of a file's name. -fn check_read_only( - mapped: &[Mapped<'_>], - write: &[PolicyPath], - write_name: &str, - roots: &[PathBuf], -) -> Result<()> { - // Write paths narrow the writes of the read-write directory that a child exports whole. - let narrowed = |canonical: &Path| { - roots.iter().any(|root| root == canonical) - && write - .iter() - .any(|path| path.canonical.starts_with(canonical)) - }; - let inside = |inner: &Mapped<'_>, outer_name: &str, outer: &Path, instead: &str| { - Err(Error::unsupported(format!( - "{} entry {} lies inside the {outer_name} entry {}, where the host cannot keep it \ - read-only; instead, {instead}", - inner.name, - inner.given.display(), - outer.display() - ))) - }; - for inner in mapped.iter().filter(|entry| entry.read_only) { - if let Some(path) = write - .iter() - .find(|path| inner.canonical.starts_with(&path.canonical)) - { - return inside( - inner, - write_name, - &path.given, - "narrow the writes to paths outside it", - ); - } - let Some(outer) = mapped.iter().find(|outer| { - !outer.read_only - && !narrowed(&outer.canonical) - && (outer.canonical == inner.canonical - || (outer.directory && inner.canonical.starts_with(&outer.canonical))) - }) else { - continue; - }; - if outer.canonical == inner.canonical { - return Err(Error::unsupported(format!( - "{} entry {} names the object of the {} entry {}, which the guest may write, so \ - the host cannot keep it read-only", - inner.name, - inner.given.display(), - outer.name, - outer.given.display() - ))); - } - return inside( - inner, - outer.name, - &outer.given, - "map that directory read-only and the paths in it that the guest writes read-write", - ); - } - Ok(()) -} - -/// Refuses roots that name one directory or file: OpenVMM refuses one object exported twice, -/// which bind mounts and hard links make possible. -fn distinct_roots(roots: &[PathBuf]) -> Result<()> { - let mut identities: Vec<(FileIdentity, &Path)> = Vec::with_capacity(roots.len()); - for root in roots { - let current = identity(root)?; - if let Some((_, other)) = identities.iter().find(|(other, _)| *other == current) { - return Err(Error::unsupported(format!( - "{} and {} are the same host directory or file; map it once", - other.display(), - root.display() - ))); - } - identities.push((current, root)); - } - Ok(()) -} - -/// Resolves the allowed paths of `list`, which must exist inside a child of `roots`, below its -/// root. -fn allowed_paths(list: PathList<'_>, roots: &[PathBuf]) -> Result> { - let field = list.name; - let mut paths: Vec = Vec::with_capacity(list.paths.len()); - for path in list.paths { - let canonical = resolve_policy_path(field, path)?; - if roots.contains(&canonical) { - return Err(Error::unsupported(format!( - "{field} entry {} is an exported path itself rather than a path inside one", - path.display() - ))); - } - if !roots.iter().any(|root| canonical.starts_with(root)) { - return Err(Error::unsupported(format!( - "{field} entry {} lies in no mapped directory", - path.display() - ))); - } - if !paths.iter().any(|other| other.canonical == canonical) { - paths.push(PolicyPath { - given: path.clone(), - canonical, - }); - } - } - Ok(paths) -} - -/// Resolves the denied paths of `list` that exist, and refuses one that hides a mapped path, -/// which `allowed` paths do not show again, or one that does not exist inside a mapped path. -fn denied_paths( - list: PathList<'_>, - mapped: &[Mapped<'_>], - allowed: &[PolicyPath], -) -> Result> { - let field = list.name; - let mut resolved = Vec::with_capacity(list.paths.len()); - for path in list.paths { - let (canonical, exists) = canonicalize_lenient(path).map_err(|error| { - Error::unsupported(format!( - "{field} entry {} cannot be resolved", - path.display() - )) - .with_source(error) - })?; - resolved.push((path, canonical, exists)); - } - let denied_paths: Vec = resolved - .iter() - .filter(|(_, _, exists)| *exists) - .map(|(_, canonical, _)| canonical.clone()) - .collect(); - let allowed_paths = canonicals(allowed); - let mut denied: Vec = Vec::with_capacity(denied_paths.len()); - for (path, canonical, exists) in resolved { - if !exists { - if let Some(entry) = mapped - .iter() - .find(|entry| canonical.starts_with(&entry.canonical)) - { - return Err(Error::unsupported(format!( - "{field} entry {} does not exist, so it cannot be hidden inside the mapped \ - path {}", - path.display(), - entry.given.display() - ))); - } - continue; - } - // A mapped directory that is denied itself shows the allowed paths inside it, and nothing - // else, whichever denied path contains it. - let hidden = mapped.iter().find(|entry| { - entry.canonical.starts_with(&canonical) - && match nearest_rule(&entry.canonical, &denied_paths, &allowed_paths, true) { - Some((rule, true)) => { - !(entry.directory - && rule == entry.canonical.as_path() - && allowed_paths.iter().any(|shown| { - *shown != entry.canonical && shown.starts_with(&entry.canonical) - })) - } - _ => false, - } - }); - if let Some(entry) = hidden { - return Err(Error::unsupported(format!( - "{field} entry {} contains the mapped path {}", - path.display(), - entry.given.display() - ))); - } - if !denied.iter().any(|other| other.canonical == canonical) { - denied.push(PolicyPath { - given: path.clone(), - canonical, - }); - } - } - Ok(denied) -} - -/// Refuses an allowed path that is also denied, or whose nearest enclosing policy path is not a -/// denied one, which OpenVMM refuses. -fn check_allowed_paths( - allowed_name: &str, - denied_name: &str, - allowed: &[PolicyPath], - denied: &[PolicyPath], -) -> Result<()> { - let (denied_paths, allowed_paths) = (canonicals(denied), canonicals(allowed)); - for path in allowed { - let refused = |problem: String| { - Err(Error::unsupported(format!( - "{allowed_name} entry {} {problem}", - path.given.display() - ))) - }; - if denied_paths.contains(&path.canonical) { - return refused(format!("is also a {denied_name} entry")); - } - match nearest_rule(&path.canonical, &denied_paths, &allowed_paths, false) { - Some((_, true)) => {} - Some((other, false)) => { - return refused(format!( - "lies inside the {allowed_name} entry {}, which shows it already", - other.display() - )); - } - None => { - return refused(format!( - "lies inside no {denied_name} entry, so it has nothing to show again" - )); - } - } - } - Ok(()) -} - -/// Refuses a write path that a denied path hides, which OpenVMM refuses. -fn check_write_paths( - write_name: &str, - denied_name: &str, - write: &[PolicyPath], - denied: &[PolicyPath], - allowed: &[PolicyPath], -) -> Result<()> { - let (denied_paths, allowed_paths) = (canonicals(denied), canonicals(allowed)); - for path in write { - if let Some((rule, true)) = - nearest_rule(&path.canonical, &denied_paths, &allowed_paths, true) - { - return Err(Error::unsupported(format!( - "{write_name} entry {} lies inside the {denied_name} entry {}, which hides it", - path.given.display(), - rule.display() - ))); - } - } - Ok(()) -} - -/// Returns the denied paths of one child that OpenVMM needs to hide, in order: those that no other -/// of them hides already, without an allowed path between them. OpenVMM refuses the others. -/// -/// It takes one child's paths alone: a denied path outside the child, such as one that contains a -/// denied root, hides none of the child for OpenVMM, which receives each child's policy alone. -fn needed_denied_paths(denied: &[PathBuf], allowed: &[PathBuf]) -> Vec { - let mut needed: Vec = denied - .iter() - .filter(|path| { - !nearest_rule(path, denied, allowed, false).is_some_and(|(_, denied)| denied) - }) - .cloned() - .collect(); - needed.sort(); - needed -} - -/// Explains how the policy of `child` breaks a rule of OpenVMM, or returns `None` if it keeps them -/// all: it lists no path twice, hides no path inside another that it hides without an allowed -/// path between them, shows each allowed path again inside a hidden path without another allowed -/// path between them, and makes writable only paths strictly inside it that it shows, none inside -/// another. [`plan`] checks these rules on the paths that it resolves. It reads nothing from the -/// host. -fn policy_rule_problem(child: &ExportChild) -> Option { - let (denied, allowed) = (&child.denied, &child.allowed); - for (paths, kind) in [ - (denied, "hidden"), - (allowed, "allowed"), - (&child.write, "writable"), - ] { - let repeated = paths - .iter() - .enumerate() - .find(|&(position, path)| paths[..position].contains(path)); - if let Some((_, path)) = repeated { - return Some(format!("it lists the {kind} path {} twice", path.display())); - } - } - for path in denied { - if let Some((rule, true)) = nearest_rule(path, denied, allowed, false) { - return Some(format!( - "it hides {} inside {}, which it hides already", - path.display(), - rule.display() - )); - } - } - for path in allowed { - if denied.contains(path) { - return Some(format!("it both hides and shows {}", path.display())); - } - match nearest_rule(path, denied, allowed, false) { - Some((_, true)) => {} - Some((rule, false)) => { - return Some(format!( - "it shows {} again inside {}, which it shows already", - path.display(), - rule.display() - )); - } - None => { - return Some(format!( - "it shows {} again, but hides no path that contains it", - path.display() - )); - } - } - } - for path in &child.write { - if *path == child.root { - return Some(format!( - "it makes all of {} writable through a writable path", - path.display() - )); - } - if let Some(other) = child - .write - .iter() - .find(|other| *other != path && path.starts_with(other)) - { - return Some(format!( - "it makes {} writable inside {}, which it makes writable already", - path.display(), - other.display() - )); - } - if let Some((rule, true)) = nearest_rule(path, denied, allowed, true) { - return Some(format!( - "it makes {} writable, but {} hides it", - path.display(), - rule.display() - )); - } - } - None -} - -/// Returns the nearest of the `denied` and `allowed` paths that contains `path`, which counts -/// itself only when `inclusive`, and whether it is a denied path: the rule by which OpenVMM -/// decides whether the guest sees `path`. -fn nearest_rule<'p>( - path: &Path, - denied: &'p [PathBuf], - allowed: &'p [PathBuf], - inclusive: bool, -) -> Option<(&'p Path, bool)> { - denied - .iter() - .map(|rule| (rule, true)) - .chain(allowed.iter().map(|rule| (rule, false))) - .filter(|(rule, _)| (inclusive || rule.as_path() != path) && path.starts_with(rule)) - .max_by_key(|(rule, _)| rule.components().count()) - .map(|(rule, denied)| (rule.as_path(), denied)) -} - -/// Whether a child is writable, and the paths to which OpenVMM limits its writes. -struct Writable { - /// Whether the guest may modify the child. - writable: bool, - /// The paths to which OpenVMM limits the guest's writes; empty if the guest may modify all of - /// a writable child. - write: Vec, -} - -impl Writable { - /// Returns whether a workload could move `path`, which lies in the child at `root`: whether it - /// lies strictly inside a writable part, in a directory that the workload may change. - fn moves(&self, root: &Path, path: &Path) -> bool { - self.writable - && path != root - && (self.write.is_empty() - || self - .write - .iter() - .any(|write| path != write && path.starts_with(write))) - } -} - -/// Returns the writable part of the child at `root`: its read-write mappings, or, when one of them -/// is the whole child, all of it, unless write paths narrow its writes to them and the read-write -/// mappings inside it. -fn writable_part(root: &Path, mapped: &[Mapped<'_>], write: &[PolicyPath]) -> Writable { - let read_write: Vec = mapped - .iter() - .filter(|entry| !entry.read_only && entry.canonical.starts_with(root)) - .map(|entry| entry.canonical.clone()) - .collect(); - if read_write.is_empty() { - return Writable { - writable: false, - write: Vec::new(), - }; - } - let narrowing: Vec = write - .iter() - .filter(|path| path.canonical.starts_with(root)) - .map(|path| path.canonical.clone()) - .collect(); - let write = if !read_write.iter().any(|path| path == root) { - // Unless a read-write mapping covers the whole child, OpenVMM limits writes to them. - outermost(read_write) - } else if narrowing.is_empty() { - Vec::new() - } else { - outermost( - narrowing - .into_iter() - .chain(read_write.into_iter().filter(|path| path != root)) - .collect(), - ) - }; - Writable { - writable: true, - write, - } -} - -/// Refuses an exported directory or file that belongs to UID 0 or GID 0, to which the caller -/// owner would squash guest root. -fn check_caller_roots(children: &[ExportChild]) -> Result<()> { - #[cfg(unix)] - for child in children { - use std::os::unix::fs::MetadataExt; - - let metadata = fs::metadata(&child.root).map_err(|error| { - Error::unsupported(format!("cannot inspect {}", child.root.display())) - .with_source(error) - })?; - if metadata.uid() == 0 || metadata.gid() == 0 { - return Err(Error::unsupported(format!( - "the caller owner squashes guest root to the owner of each exported directory or \ - file, so {} must not be owned by UID 0 or GID 0", - child.root.display() - ))); - } - } - #[cfg(not(unix))] - let _ = children; - Ok(()) -} - -/// Explains why a mapped host directory, given in canonical form, cannot be exported, or returns -/// `None` if it can: a whole volume. It inspects only the path, not the host. -fn volume_root_refusal(canonical: &Path) -> Option<&'static str> { - (!canonical - .components() - .any(|component| matches!(component, Component::Normal(_)))) - .then_some( - "is a whole volume, which the openvmm backend does not map; map the directories inside \ - it", - ) -} - -/// Explains why `path`, a host path of a recorded plan, is not in the form in which [`plan`] -/// resolves paths, or returns `None` if it is: without a NUL character, which no name holds and -/// no process argument carries; without a `..` component, through which a path that lies inside a -/// directory by its text could leave it; and, on Windows, on a local drive, without the verbatim -/// prefix. It inspects only the path, not the host. -fn resolved_path_problem(path: &Path) -> Option<&'static str> { - if path.as_os_str().as_encoded_bytes().contains(&0) { - return Some("contains a NUL character"); - } - if path - .components() - .any(|component| component == Component::ParentDir) - { - return Some("has a `..` component"); - } - if cfg!(windows) - && !matches!( - path.components().next(), - Some(Component::Prefix(prefix)) if matches!(prefix.kind(), Prefix::Disk(_)) - ) - { - return Some("is not a path on a local drive"); - } - None -} - -/// Sorts `paths`, drops duplicates, and returns those that no other of them contains. -fn outermost(mut paths: Vec) -> Vec { - paths.sort(); - paths.dedup(); - paths - .iter() - .filter(|path| { - !paths - .iter() - .any(|other| other != *path && path.starts_with(other)) - }) - .cloned() - .collect() -} - -impl ExportPlan { - /// Checks that a plan read back from a record is one that [`plan`] could have made for a root - /// that reserves `reserved`: at least one and at most [`MAX_CHILDREN`] children, none inside - /// another, each an absolute host path below a volume's root, with its host paths in the form - /// in which `plan` resolves them, whose policy paths lie inside it within OpenVMM's limits and - /// keep its rules, with an identity slot for each denied path, and writable exactly when a bind - /// maps a path of it read-write; and at least one and at most [`MAX_HOST_MAPPINGS`] binds, - /// each of a known child, from a relative source that holds the host's names joined with `/`, - /// that a guest path can hold, and that the child does not hide, unless it shows paths inside - /// it again, to a free guest target, parents before children, with a bind of every child's - /// root. A target inside another's must show what the other shows there. A child's write - /// paths must be the outermost paths of its read-write binds, unless one maps the whole child, - /// when they may narrow its writes only to paths that keep its other read-write binds - /// writable; and each denied or mapped object that a workload could move, strictly inside a - /// part of the child that the guest may write, must carry its identity. It reads nothing from - /// the host; [`verify`](Self::verify) checks the host. - /// - /// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported). - pub fn check(&self, reserved: ReservedTargets<'_>) -> Result<()> { - let invalid = |reason: String| { - Err(Error::unsupported(format!( - "the recorded export plan is invalid: {reason}" - ))) - }; - if self.children.is_empty() || self.children.len() > MAX_CHILDREN { - return invalid(format!( - "it exports {} directories and files, but OpenVMM exports from 1 to \ - {MAX_CHILDREN}", - self.children.len() - )); - } - if self.binds.is_empty() || self.binds.len() > MAX_HOST_MAPPINGS { - return invalid(format!( - "it maps {} paths, but a sandbox maps from 1 to {MAX_HOST_MAPPINGS}", - self.binds.len() - )); - } - let mut policy_bytes = 0; - for (index, child) in self.children.iter().enumerate() { - if !child.root.is_absolute() || volume_root_refusal(&child.root).is_some() { - return invalid(format!( - "child {index}, {}, is not an absolute path below a volume's root", - child.root.display() - )); - } - if let Some((path, problem)) = std::iter::once(&child.root) - .chain(&child.denied) - .chain(&child.allowed) - .chain(&child.write) - .find_map(|path| resolved_path_problem(path).map(|problem| (path, problem))) - { - return invalid(format!( - "child {index} has the host path {path:?}, which {problem}" - )); - } - if child.denied_identities.len() != child.denied.len() { - return invalid(format!( - "child {index} has {} denied paths but {} identities", - child.denied.len(), - child.denied_identities.len() - )); - } - if let Some(path) = [&child.denied, &child.allowed, &child.write] - .into_iter() - .flatten() - .find(|path| !path.starts_with(&child.root) || (child.file && *path != &child.root)) - { - return invalid(format!( - "child {index} has the policy path {}, which is not inside it", - path.display() - )); - } - for (paths, action) in [ - (&child.denied, "hide"), - (&child.allowed, "show again"), - (&child.write, "make writable"), - ] { - match policy_limits(&child.root, paths, action) { - Ok(bytes) => policy_bytes += bytes, - Err(problem) => return invalid(format!("child {index}: {problem}")), - } - } - if let Some(other) = self.children[..index].iter().position(|other| { - other.root.starts_with(&child.root) || child.root.starts_with(&other.root) - }) { - return invalid(format!( - "child {index}, {}, overlaps child {other}, {}", - child.root.display(), - self.children[other].root.display() - )); - } - if let Some(problem) = policy_rule_problem(child) { - return invalid(format!("child {index}: {problem}")); - } - if !child.writable && !child.write.is_empty() { - return invalid(format!("child {index} is read-only but has writable paths")); - } - if !self - .binds - .iter() - .any(|bind| bind.child == index && bind.source.is_empty()) - { - return invalid(format!("it maps no path at the root of child {index}")); - } - // OpenVMM lets the guest write a child that a read-write bind needs, and no other. - let read_write = self - .binds - .iter() - .any(|bind| bind.child == index && !bind.read_only); - if child.writable != read_write { - return invalid(if child.writable { - format!("child {index} is writable but maps no path read-write") - } else { - format!("child {index} is read-only but maps a path read-write") - }); - } - } - if policy_bytes > MAX_AGGREGATE_POLICY_BYTES { - return invalid(format!( - "the paths that OpenVMM is to hide, show again, or make writable exceed its \ - {MAX_AGGREGATE_POLICY_BYTES}-byte limit" - )); - } - let mut previous: Option<&Bind> = None; - for (index, bind) in self.binds.iter().enumerate() { - let Some(child) = self.children.get(bind.child) else { - return invalid(format!( - "it maps a path of the unknown child {}", - bind.child - )); - }; - if !bind.source.is_empty() - && (child.file - || bind - .source - .split('/') - .any(|component| matches!(component, "" | "." | ".."))) - { - return invalid(format!( - "it maps the path {:?} of child {}, which is not a path inside a directory", - bind.source, bind.child - )); - } - // The guest receives the source as a path below its mount of the export. - if bind.source.contains('\0') { - return invalid(format!( - "it maps the path {:?} of child {}, which contains a NUL character", - bind.source, bind.child - )); - } - if GUEST_EXPORT.len() + 1 + bind.guest_source().len() > MAX_GUEST_PATH { - return invalid(format!( - "it maps a path of {} bytes in child {}, which is too long for a guest path", - bind.source.len(), - bind.child - )); - } - let path = if bind.source.is_empty() { - child.root.clone() - } else { - child.root.join(&bind.source) - }; - // The host checks the object at that path, which must be the one that the guest maps: - // the source must hold the host's names below the root, joined with `/`, as `plan` - // spells them. A Windows host would read a backslash in a name as a separator, and a - // name with a root or a drive as another path. - if relative_text(&child.root, &path).as_deref() != Some(bind.source.as_str()) { - return invalid(format!( - "it maps the path {:?} of child {}, which the host reads as other names", - bind.source, bind.child - )); - } - // A mapped path shows something: OpenVMM does not hide it, or it shows the allowed - // paths inside it again. - if matches!( - nearest_rule(&path, &child.denied, &child.allowed, true), - Some((_, true)) - ) && !child - .allowed - .iter() - .any(|shown| *shown != path && shown.starts_with(&path)) - { - return invalid(format!( - "it maps {} at {}, but child {} hides it", - path.display(), - bind.target, - bind.child - )); - } - if let Some(problem) = target_problem(&bind.target) { - return invalid(format!("its guest target {:?} {problem}", bind.target)); - } - if let Some(reserved) = reserved.reservation(&bind.target) { - return invalid(format!( - "its guest target {} would cover the guest's {reserved}", - bind.target - )); - } - if let Some(previous) = previous - && (depth(&previous.target), previous.target.as_str()) - >= (depth(&bind.target), bind.target.as_str()) - { - return invalid(format!( - "it maps {} after {}, out of order or twice", - bind.target, previous.target - )); - } - // A target inside another shows what the other shows there, as `plan` requires. - for outer in &self.binds[..index] { - let Some(rest) = bind - .target - .strip_prefix(outer.target.as_str()) - .and_then(|rest| rest.strip_prefix('/')) - else { - continue; - }; - let shown = if outer.source.is_empty() { - rest.to_owned() - } else { - format!("{}/{rest}", outer.source) - }; - if bind.child != outer.child || bind.source != shown { - return invalid(format!( - "it maps {} inside {}, where it would hide part of that path", - bind.target, outer.target - )); - } - } - previous = Some(bind); - } - for (index, child) in self.children.iter().enumerate() { - let bound = |bind: &Bind| { - if bind.source.is_empty() { - child.root.clone() - } else { - child.root.join(&bind.source) - } - }; - let read_write: Vec = self - .binds - .iter() - .filter(|bind| bind.child == index && !bind.read_only) - .map(bound) - .collect(); - // A child that a read-write bind maps whole is writable throughout, unless write paths - // narrow its writes, which must then keep its other read-write binds writable; any - // other child's writes are narrowed to exactly what its read-write binds map. - if read_write.contains(&child.root) { - if let Some(path) = read_write.iter().find(|path| { - **path != child.root - && !child.write.is_empty() - && !child.write.iter().any(|write| path.starts_with(write)) - }) { - return invalid(format!( - "child {index} narrows its writes so that {}, which it maps read-write, \ - stays read-only", - path.display() - )); - } - } else { - let mut write = child.write.clone(); - write.sort(); - if outermost(read_write) != write { - return invalid(format!( - "child {index} lets the guest write other paths than those that it maps \ - read-write" - )); - } - } - // A workload that may write a part of the child could move an object there and put a - // decoy at its path, so the plan pins each denied and mapped object in such a part, as - // `plan` does, for `verify` to check it at the next start. - let part = Writable { - writable: child.writable, - write: child.write.clone(), - }; - if let Some((path, _)) = child - .denied - .iter() - .zip(&child.denied_identities) - .find(|(path, identity)| identity.is_none() && part.moves(&child.root, path)) - { - return invalid(format!( - "child {index} does not pin the denied path {}, which a workload could move", - path.display() - )); - } - if let Some(bind) = self.binds.iter().find(|bind| { - bind.child == index - && bind.identity.is_none() - && part.moves(&child.root, &bound(bind)) - }) { - return invalid(format!( - "it does not pin the object that it maps at {}, which a workload could move", - bind.target - )); - } - } - Ok(()) - } - - /// Checks that the host keeps every read-only mapping read-only, that every child's root is - /// still the kind of object that [`plan`] exported, that every mapped and denied path inside a - /// writable part still names the object it named then, and that the host still supports the - /// owner. - /// - /// [`plan`] refuses a read-only path inside a part that the guest may write, which only the - /// guest's bind mount would keep read-only; a recorded plan that holds one fails with - /// [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported). A file that a directory replaced - /// would otherwise grant the whole directory, and a workload that may write a directory could - /// rename a denied or mapped object inside it and put a decoy at its path, so the next start - /// would hide or bind the decoy instead; these fail with - /// [`ErrorKind::Other`](crate::ErrorKind::Other). With the caller owner, the host must be a - /// Linux host, and no exported directory or file may belong to UID 0 or GID 0; these fail with - /// [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported). Run it while the VM is stopped, - /// so that no workload can race with it; OpenVMM checks each child's kind again when it opens - /// it. - pub fn verify(&self) -> Result<()> { - for bind in self.binds.iter().filter(|bind| bind.read_only) { - let (child, path) = self.bound(bind)?; - // Without writable paths, the guest may write all of a writable child. - if child.writable - && (child.write.is_empty() - || child.write.iter().any(|write| path.starts_with(write))) - { - return Err(Error::unsupported(format!( - "the sandbox maps {} read-only inside a part of {} that the guest may write, \ - where the host cannot keep it read-only; deprovision the sandbox, and \ - provision it again without read-only paths inside read-write directories", - path.display(), - child.root.display() - ))); - } - } - let changed = |path: &Path| { - Err(Error::other(format!( - "{} no longer names the host object it named at provision; deprovision the \ - sandbox and provision it again", - path.display() - ))) - }; - for child in &self.children { - let kind_matches = fs::symlink_metadata(&child.root).is_ok_and(|metadata| { - if child.file { - metadata.is_file() - } else { - metadata.is_dir() - } - }); - if !kind_matches { - return changed(&child.root); - } - } - for bind in &self.binds { - let Some(expected) = bind.identity else { - continue; - }; - let (_, path) = self.bound(bind)?; - match platform::file_identity(&path) { - Ok((device, index)) if (FileIdentity { device, index }) == expected => {} - _ => return changed(&path), - } - } - for child in &self.children { - for (path, expected) in child.denied.iter().zip(&child.denied_identities) { - let Some(expected) = expected else { - continue; - }; - match platform::file_identity(path) { - Ok((device, index)) if (FileIdentity { device, index }) == *expected => {} - _ => return changed(path), - } - } - } - if self.owner == Owner::Caller { - if cfg!(windows) { - return Err(Error::unsupported(CALLER_OWNER_ON_WINDOWS)); - } - check_caller_roots(&self.children)?; - } - Ok(()) - } - - /// Returns the child that holds `bind` and the host path that `bind` maps. - fn bound(&self, bind: &Bind) -> Result<(&ExportChild, PathBuf)> { - let Some(child) = self.children.get(bind.child) else { - return Err(Error::io( - "the sandbox state maps a path of an unknown host directory", - )); - }; - let path = if bind.source.is_empty() { - child.root.clone() - } else { - child.root.join(&bind.source) - }; - Ok((child, path)) - } - - /// Kernel command-line token that tells the guest agent how many bind mounts to expect over - /// the control channel. - pub fn kernel_token(&self) -> String { - format!("{MAPS_TOKEN}{}", self.binds.len()) - } - - /// The bind mounts in the order and form in which the guest agent receives them. - pub fn guest_table(&self) -> Vec { - self.binds - .iter() - .map(|bind| HostMap { - source: bind.guest_source(), - target: bind.target.clone(), - read_only: bind.read_only, - }) - .collect() - } - - /// OpenVMM options that export the plan's directories and files at [`GUEST_EXPORT`], with - /// their access policies and the owner. A file child carries the `file` flag, so OpenVMM - /// refuses a directory that replaced the file. - pub fn openvmm_arguments(&self) -> Vec { - let mut arguments = vec!["--mount-aggregate".to_owned(), GUEST_EXPORT.to_owned()]; - for (index, child) in self.children.iter().enumerate() { - let mode = if child.writable { "rw" } else { "ro" }; - let kind = if child.file { ",file" } else { "" }; - arguments.push("--mount-child".to_owned()); - arguments.push(format!( - "{index},{},{mode}{kind}", - display_path(&child.root) - )); - } - for (option, paths) in [ - ( - "--mount-deny", - ExportChild::denied_paths as fn(&ExportChild) -> &[PathBuf], - ), - ("--mount-allow", ExportChild::allowed_paths), - ("--mount-write", ExportChild::write_paths), - ] { - for path in self.children.iter().flat_map(paths) { - arguments.push(option.to_owned()); - arguments.push(display_path(path)); - } - } - if self.owner == Owner::Caller { - arguments.extend(["--mount-owner".to_owned(), "caller".to_owned()]); - } - arguments - } -} - -impl ExportChild { - /// Returns the paths that OpenVMM hides. - fn denied_paths(&self) -> &[PathBuf] { - &self.denied - } - - /// Returns the paths that stay visible inside denied paths. - fn allowed_paths(&self) -> &[PathBuf] { - &self.allowed - } - - /// Returns the paths to which OpenVMM limits the guest's writes. - fn write_paths(&self) -> &[PathBuf] { - &self.write - } -} - -/// Applies OpenVMM's rules for exported directories, which it reaches without links. -fn check_root(root: &Path) -> Result<()> { - let mut current = PathBuf::new(); - for component in root.components() { - current.push(component.as_os_str()); - if !matches!(component, Component::Normal(_)) { - continue; - } - let metadata = fs::symlink_metadata(¤t).map_err(|error| { - Error::unsupported(format!("cannot inspect {}", current.display())).with_source(error) - })?; - if metadata.file_type().is_symlink() || is_reparse_point(&metadata) { - return Err(Error::unsupported(format!( - "OpenVMM cannot export {}, because {} is a link or reparse point", - root.display(), - current.display() - ))); - } - } - Ok(()) -} - -/// Applies OpenVMM's rules for the paths that it is to `action` in the exported directory `root`, -/// so a policy that OpenVMM would refuse fails at provision rather than at every start: those of -/// [`policy_limits`], and on the host, no link or reparse point below `root` and no other file -/// system. Returns their combined length. -fn check_policy_paths(root: &Path, paths: &[PathBuf], action: &str) -> Result { - let unsupported = |message: String| Err(Error::unsupported(message)); - let total = policy_limits(root, paths, action).map_err(Error::unsupported)?; - for path in paths { - // `policy_limits` read each path as text below `root` already. - let relative = relative_text(root, path).unwrap_or_default(); - // The exported directory itself, which a denied root names, was checked as a root. - let components = relative - .split('/') - .filter(|component| !component.is_empty()); - let mut current = root.to_path_buf(); - for component in components { - current.push(component); - let metadata = fs::symlink_metadata(¤t).map_err(|error| { - Error::unsupported(format!("cannot inspect {}", current.display())) - .with_source(error) - })?; - if metadata.file_type().is_symlink() || is_reparse_point(&metadata) { - return unsupported(format!( - "OpenVMM cannot {action} {}, because {} is a link or reparse point", - path.display(), - current.display() - )); - } - } - #[cfg(unix)] - if platform::file_identity(path).ok().map(|(device, _)| device) - != platform::file_identity(root).ok().map(|(device, _)| device) - { - return unsupported(format!( - "OpenVMM cannot {action} {}, which lies on another file system than {}", - path.display(), - root.display() - )); - } - } - Ok(total) -} - -/// Applies those of OpenVMM's rules for the paths that it is to `action` in the exported directory -/// `root` that read nothing from the host: at most [`MAX_POLICY_PATHS`] paths, each valid UTF-8 -/// below `root`, of at most [`MAX_POLICY_PATH_BYTES`] bytes there, without colons, backslashes, -/// whitespace other than spaces, or a name that begins or ends with a space, and of at most -/// [`MAX_POLICY_BYTES`] bytes together. Returns their combined length, or why OpenVMM would refuse -/// them. -fn policy_limits( - root: &Path, - paths: &[PathBuf], - action: &str, -) -> std::result::Result { - let count = paths.len(); - if count > MAX_POLICY_PATHS { - return Err(format!( - "OpenVMM can {action} at most {MAX_POLICY_PATHS} paths in one exported directory, \ - but {} needs {count}", - root.display() - )); - } - let mut total = 0; - for path in paths { - let Some(relative) = relative_text(root, path) else { - return Err(format!("{} is not valid UTF-8", path.display())); - }; - // A name may contain spaces, but not begin or end with one. - if relative.len() > MAX_POLICY_PATH_BYTES - || relative.chars().any(|character| { - (character.is_whitespace() && character != ' ') || matches!(character, ':' | '\\') - }) - || relative - .split('/') - .any(|name| name.starts_with(' ') || name.ends_with(' ')) - { - return Err(format!( - "OpenVMM cannot {action} {}: below the exported directory {}, a path must not \ - contain colons, backslashes, or whitespace other than spaces, and no name may \ - begin or end with a space", - path.display(), - root.display() - )); - } - total += relative.len(); - } - if total > MAX_POLICY_BYTES { - return Err(format!( - "the paths that OpenVMM is to {action} in {} exceed its {MAX_POLICY_BYTES}-byte \ - limit", - root.display() - )); - } - Ok(total) -} - -#[cfg(windows)] -/// Returns whether Windows marks `metadata` as a reparse point. -pub(crate) fn is_reparse_point(metadata: &fs::Metadata) -> bool { - use std::os::windows::fs::MetadataExt; - metadata.file_attributes() & 0x400 != 0 -} - -#[cfg(not(windows))] -/// Reports no reparse points on hosts whose metadata has no such Windows attribute. -pub(crate) fn is_reparse_point(_metadata: &fs::Metadata) -> bool { - false -} - -/// Returns the stable device-and-file identity of `path` for rename detection. -fn identity(path: &Path) -> Result { - platform::file_identity(path) - .map(|(device, index)| FileIdentity { device, index }) - .map_err(|error| { - Error::unsupported(format!("cannot identify {}", path.display())).with_source(error) - }) -} - -/// Path of `path` relative to `root`, `/`-separated; empty for the root itself. -fn relative_text(root: &Path, path: &Path) -> Option { - let relative = path.strip_prefix(root).ok()?; - let mut parts = Vec::new(); - for component in relative.components() { - parts.push(component.as_os_str().to_str()?); - } - Some(parts.join("/")) -} - -/// Canonicalized, validated form of one requested host mapping. -struct Mapped<'a> { - /// How messages name the list or option that requested it. - name: &'a str, - /// Host path exactly as the caller supplied it, for diagnostics. - given: PathBuf, - /// Link-resolved host path used to construct the export. - canonical: PathBuf, - /// Absolute guest destination of the bind mount. - target: String, - /// Whether the guest must mount this path read-only. - read_only: bool, - /// Whether the mapped host object is a directory rather than a regular file. - directory: bool, -} - -/// Returns whether the guest path `path` is `directory` or one of its descendants, comparing -/// whole components: `/run/nvx` is within `/run`, but `/runner` is not, and every absolute path is -/// within the root, `/`. -pub(crate) fn within(path: &str, directory: &str) -> bool { - path == directory - || path - .strip_prefix(directory) - .is_some_and(|rest| directory.ends_with('/') || rest.starts_with('/')) -} - -/// Counts the nonempty components of an absolute guest path. -fn depth(path: &str) -> usize { - path.split('/').filter(|part| !part.is_empty()).count() -} - -/// Resolves links and returns a path without the Windows verbatim prefix, which must still name -/// the object that the resolved path names. -/// -/// Windows reads a path without the prefix differently: it drops a trailing period or space of a -/// name, among other things. A path that resolves to itself again names the same object; one that -/// only the verbatim prefix keeps, such as that of a directory whose name ends in a period, would -/// name another object, or none, so it is refused. -pub(crate) fn canonicalize(path: &Path) -> std::io::Result { - let canonical = fs::canonicalize(path)?; - let Some(stripped) = strip_verbatim(&canonical) else { - return Err(std::io::Error::new( - std::io::ErrorKind::Unsupported, - "network paths are unsupported", - )); - }; - if stripped != canonical && !fs::canonicalize(&stripped).is_ok_and(|again| again == canonical) { - return Err(std::io::Error::new( - std::io::ErrorKind::Unsupported, - "only a verbatim path names it, as with a name that ends in a period or a space", - )); - } - Ok(stripped) -} - -/// Canonicalizes the deepest existing ancestor and appends the rest. Returns whether the -/// complete path exists. -fn canonicalize_lenient(path: &Path) -> std::io::Result<(PathBuf, bool)> { - let mut existing = path.to_path_buf(); - let mut missing = Vec::new(); - loop { - match canonicalize(&existing) { - Ok(canonical) => { - let exists = missing.is_empty(); - let mut resolved = canonical; - for component in missing.iter().rev() { - resolved.push(component); - } - return Ok((resolved, exists)); - } - Err(error) if error.kind() == std::io::ErrorKind::NotFound => { - let Some(name) = existing.file_name().map(ToOwned::to_owned) else { - return Err(error); - }; - missing.push(name); - if !existing.pop() { - return Err(error); - } - } - Err(error) => return Err(error), - } - } -} - -#[cfg(windows)] -/// Removes a local Windows verbatim prefix and rejects UNC paths. -fn strip_verbatim(path: &Path) -> Option { - let text = path.to_str()?; - match text.strip_prefix(r"\\?\") { - Some(rest) if rest.starts_with("UNC\\") => None, - Some(rest) => Some(PathBuf::from(rest)), - None if text.starts_with(r"\\") => None, - None => Some(path.to_path_buf()), - } -} - -#[cfg(not(windows))] -/// Returns the unchanged path on hosts without Windows verbatim prefixes. -fn strip_verbatim(path: &Path) -> Option { - Some(path.to_path_buf()) -} - -/// Returns a lossily decoded path for user-facing diagnostics. -fn display_path(path: &Path) -> String { - path.to_string_lossy().into_owned() -} - -#[cfg(test)] -mod tests { - use super::*; - use crate::ErrorKind; - - /// The lists of a filesystem policy as callers hold them, under the names that messages give - /// them. - #[derive(Debug, Default)] - struct FilesystemPolicy { - readonly_paths: Vec, - readwrite_paths: Vec, - denied_paths: Vec, - } - - fn plan(policy: &FilesystemPolicy) -> Result> { - let mappings: Vec> = Mapping::convention( - PathList { - name: "filesystem.readonlyPaths", - paths: &policy.readonly_paths, - }, - true, - ) - .chain(Mapping::convention( - PathList { - name: "filesystem.readwritePaths", - paths: &policy.readwrite_paths, - }, - false, - )) - .collect(); - super::plan(&ExportRequest { - denied: PathList { - name: "filesystem.deniedPaths", - paths: &policy.denied_paths, - }, - ..ExportRequest::new(&mappings, ReservedTargets::BLOCKLESS_ROOT) - }) - } - - fn root() -> tempfile::TempDir { - let directory = tempfile::tempdir().unwrap(); - for name in [ - "work/src/secret", - "work/out", - "work/private", - "projects/app", - "projects/private", - "build-output", - "tools/bin", - ] { - fs::create_dir_all(directory.path().join(name)).unwrap(); - } - fs::write(directory.path().join("work/config.json"), b"{}").unwrap(); - fs::write(directory.path().join("work/run.sh"), b"true").unwrap(); - directory - } - - fn canonical(path: &Path) -> PathBuf { - canonicalize(path).unwrap() - } - - fn sources(mapping: &ExportPlan) -> Vec<(String, bool)> { - mapping - .binds - .iter() - .map(|bind| (bind.guest_source(), bind.read_only)) - .collect() - } - - fn option_values(arguments: &[String], option: &str) -> Vec { - arguments - .windows(2) - .filter(|pair| pair[0] == option) - .map(|pair| pair[1].clone()) - .collect() - } - - #[test] - fn guest_paths_follow_the_mxc_convention() { - if cfg!(windows) { - assert_eq!( - guest_path(Path::new(r"C:\Work\src")).as_deref(), - Some("/mnt/c/Work/src") - ); - assert_eq!(guest_path(Path::new(r"D:\")).as_deref(), Some("/mnt/d")); - assert_eq!( - guest_path(Path::new(r"\\?\C:\x\.\y")).as_deref(), - Some("/mnt/c/x/y") - ); - assert_eq!(guest_path(Path::new(r"\\server\share\x")), None); - assert_eq!(guest_path(Path::new(r"\x")), None); - } else { - assert_eq!( - guest_path(Path::new("/home/me/src")).as_deref(), - Some("/home/me/src") - ); - } - assert_eq!(guest_path(Path::new("relative")), None); - assert_eq!(guest_path(Path::new("..")), None); - } - - /// Verifies that a guest path is within itself and its ancestors only, comparing whole - /// components, as the reserved targets here and the sandbox's share checks require. - #[test] - fn guest_paths_are_within_themselves_and_their_ancestors() { - for (path, directory) in [ - ("/run", "/run"), - ("/run/nvx", "/run"), - ("/run/nvx/t", "/run"), - ("/", "/"), - ("/run", "/"), - ("/run/nvx", "/"), - ] { - assert!(within(path, directory), "{path} {directory}"); - } - for (path, directory) in [ - ("/runner", "/run"), - ("/run", "/run/nvx"), - ("/srv/run", "/run"), - ("/", "/run"), - ] { - assert!(!within(path, directory), "{path} {directory}"); - } - } - - /// The layout of microsoft/nvx#281: directories that share no directory but the volume's - /// root, each with its own access. - #[test] - fn unrelated_directories_are_exported_with_their_own_access() { - let directory = root(); - let base = directory.path(); - let policy = FilesystemPolicy { - readonly_paths: vec![base.join("tools")], - readwrite_paths: vec![base.join("projects/app"), base.join("build-output")], - ..FilesystemPolicy::default() - }; - let mapping = plan(&policy).unwrap().unwrap(); - let children: Vec<(PathBuf, bool)> = mapping - .children - .iter() - .map(|child| (child.root.clone(), child.writable)) - .collect(); - assert_eq!( - children, - [ - (canonical(&base.join("build-output")), true), - (canonical(&base.join("projects/app")), true), - (canonical(&base.join("tools")), false), - ] - ); - assert!(mapping.children.iter().all(|child| child.write.is_empty() - && child.allowed.is_empty() - && child.denied.is_empty())); - // Parents mount before children, so the shallower targets come first. - assert_eq!( - sources(&mapping), - [ - ("0".to_owned(), false), - ("2".to_owned(), true), - ("1".to_owned(), false) - ] - ); - - let arguments = mapping.openvmm_arguments(); - assert_eq!(arguments[..2], ["--mount-aggregate", GUEST_EXPORT]); - assert_eq!( - option_values(&arguments, "--mount-child"), - [ - format!( - "0,{},rw", - display_path(&canonical(&base.join("build-output"))) - ), - format!( - "1,{},rw", - display_path(&canonical(&base.join("projects/app"))) - ), - format!("2,{},ro", display_path(&canonical(&base.join("tools")))), - ] - ); - assert_eq!(arguments.len(), 8); - assert_eq!(mapping.kernel_token(), "nvx_maps=3"); - let table = mapping.guest_table(); - assert_eq!(table.len(), 3); - assert_eq!( - table[1].target, - guest_path(&canonical(&base.join("tools"))).unwrap() - ); - assert!(table[1].read_only); - } - - /// The layout of microsoft/nvx#283: directories whose names hold commas, each with a denied - /// subdirectory. - #[test] - fn host_paths_may_contain_commas() { - let directory = root(); - let base = directory.path(); - let alpha = base.join("projects/alpha,beta"); - let reports = base.join("reports/2026, final"); - for path in [&alpha, &reports] { - fs::create_dir_all(path.join("secret")).unwrap(); - } - let policy = FilesystemPolicy { - readonly_paths: vec![reports.clone()], - readwrite_paths: vec![alpha.clone()], - denied_paths: vec![alpha.join("secret"), reports.join("secret")], - }; - let mapping = plan(&policy).unwrap().unwrap(); - let children: Vec<(PathBuf, bool, Vec)> = mapping - .children - .iter() - .map(|child| (child.root.clone(), child.writable, child.denied.clone())) - .collect(); - assert_eq!( - children, - [ - ( - canonical(&alpha), - true, - vec![canonical(&alpha.join("secret"))] - ), - ( - canonical(&reports), - false, - vec![canonical(&reports.join("secret"))] - ), - ] - ); - // OpenVMM reads each child's mode after the last comma, so the host paths keep theirs. - let arguments = mapping.openvmm_arguments(); - assert_eq!( - option_values(&arguments, "--mount-child"), - [ - format!("0,{},rw", display_path(&canonical(&alpha))), - format!("1,{},ro", display_path(&canonical(&reports))), - ] - ); - assert_eq!( - option_values(&arguments, "--mount-deny"), - [ - display_path(&canonical(&alpha.join("secret"))), - display_path(&canonical(&reports.join("secret"))), - ] - ); - // So do the guest paths, which the control channel carries with their lengths. - let mut table: Vec<(String, bool)> = mapping - .guest_table() - .into_iter() - .map(|map| (map.target, map.read_only)) - .collect(); - table.sort(); - assert_eq!( - table, - [ - (guest_path(&canonical(&alpha)).unwrap(), false), - (guest_path(&canonical(&reports)).unwrap(), true), - ] - ); - assert!(table[0].0.ends_with("/projects/alpha,beta"), "{table:?}"); - assert!(table[1].0.ends_with("/reports/2026, final"), "{table:?}"); - } - - #[test] - fn mapped_files_are_exported_alone() { - let directory = root(); - let base = directory.path(); - let policy = FilesystemPolicy { - readonly_paths: vec![base.join("work/config.json"), base.join("work/src")], - readwrite_paths: vec![base.join("work/run.sh"), base.join("work/out")], - denied_paths: vec![ - // Not exported at all: no child holds it. - base.join("work/private"), - base.join("work/src/secret"), - ], - }; - let mapping = plan(&policy).unwrap().unwrap(); - let work = canonical(&base.join("work")); - // Each file is a child of its own, without its directory, beside the directories. - let children: Vec<(PathBuf, bool, bool)> = mapping - .children - .iter() - .map(|child| (child.root.clone(), child.file, child.writable)) - .collect(); - assert_eq!( - children, - [ - (work.join("config.json"), true, false), - (work.join("out"), false, true), - (work.join("run.sh"), true, true), - (work.join("src"), false, false), - ] - ); - assert!( - mapping - .children - .iter() - .all(|child| child.allowed.is_empty() && child.write.is_empty()) - ); - assert_eq!( - mapping.children[3].denied, - [work.join("src").join("secret")] - ); - assert_eq!( - sources(&mapping), - [ - ("0".to_owned(), true), - ("1".to_owned(), false), - ("2".to_owned(), false), - ("3".to_owned(), true), - ] - ); - - // OpenVMM exposes each file as the child itself, so nothing hides its directory, and the - // `file` flag makes it refuse a directory in the file's place. - let arguments = mapping.openvmm_arguments(); - assert_eq!( - option_values(&arguments, "--mount-child")[0], - format!("0,{},ro,file", display_path(&work.join("config.json"))) - ); - assert_eq!( - option_values(&arguments, "--mount-deny"), - [display_path(&work.join("src").join("secret"))] - ); - assert!(option_values(&arguments, "--mount-allow").is_empty()); - assert!(option_values(&arguments, "--mount-write").is_empty()); - - // A record without file children leaves the field out, and one with them names it. - let record = serde_json::to_value(&mapping).unwrap(); - assert_eq!(record["children"][0]["file"], true); - assert!(record["children"][1].get("file").is_none()); - let restored: ExportPlan = serde_json::from_value(record).unwrap(); - assert_eq!(restored, mapping); - - // A file inside a mapped directory belongs to that directory's child, whose writes - // OpenVMM limits to the file when the directory is read-only. - let nested = FilesystemPolicy { - readonly_paths: vec![base.join("work")], - readwrite_paths: vec![base.join("work/config.json")], - ..FilesystemPolicy::default() - }; - let mapping = plan(&nested).unwrap().unwrap(); - assert_eq!(mapping.children.len(), 1); - assert!(!mapping.children[0].file && mapping.children[0].writable); - assert_eq!(mapping.children[0].write, [work.join("config.json")]); - assert_eq!( - sources(&mapping), - [("0".to_owned(), true), ("0/config.json".to_owned(), false)] - ); - - // One file reached through two names is one object, which is mapped once. - let link = base.join("work/config-link.json"); - fs::hard_link(base.join("work/config.json"), &link).unwrap(); - let aliased = FilesystemPolicy { - readonly_paths: vec![base.join("work/config.json"), link], - ..FilesystemPolicy::default() - }; - let error = plan(&aliased).unwrap_err(); - assert!( - error.message().contains("same host directory or file"), - "{error}" - ); - } - - #[test] - fn policies_beyond_the_children_of_an_aggregate_are_refused() { - let directory = root(); - let base = directory.path(); - let files = |parent: &str, count: usize| -> Vec { - (0..count) - .map(|index| { - let path = base.join(parent).join(format!("file-{index:03}")); - fs::write(&path, b"x").unwrap(); - path - }) - .collect() - }; - // Up to 256 files are children of their own, however many share a directory. - let policy = FilesystemPolicy { - readonly_paths: files("tools", 256), - ..FilesystemPolicy::default() - }; - let mapping = plan(&policy).unwrap().unwrap(); - assert_eq!(mapping.children.len(), 256); - assert!( - mapping - .children - .iter() - .all(|child| child.file && child.allowed.is_empty()) - ); - - // More are refused rather than exported through their directory, which would grant - // more than the files. - let mut readonly_paths = files("tools", 256); - readonly_paths.extend(files("build-output", 1)); - let error = plan(&FilesystemPolicy { - readonly_paths, - ..FilesystemPolicy::default() - }) - .unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported); - assert!(error.message().contains("at most 256"), "{error}"); - assert!( - error - .message() - .contains("map directories that contain the files instead"), - "{error}" - ); - - // A policy that grants the directory instead needs one child. - let mut readonly_paths = vec![base.join("tools")]; - readonly_paths.extend(files("build-output", 1)); - let mapping = plan(&FilesystemPolicy { - readonly_paths, - ..FilesystemPolicy::default() - }) - .unwrap() - .unwrap(); - assert_eq!(mapping.children.len(), 2); - } - - #[test] - fn nested_mappings_merge_into_the_outermost_directory() { - let directory = root(); - let base = directory.path(); - fs::create_dir_all(base.join("work/out/cache")).unwrap(); - let policy = FilesystemPolicy { - readonly_paths: vec![base.join("work"), base.join("work/src")], - readwrite_paths: vec![base.join("work/out"), base.join("work/out/cache")], - ..FilesystemPolicy::default() - }; - let mapping = plan(&policy).unwrap().unwrap(); - assert_eq!(mapping.children.len(), 1); - let child = &mapping.children[0]; - let work = canonical(&base.join("work")); - assert!(child.writable); - // The read-only root is read-only on the host, except the outermost read-write mapping - // inside it, to which OpenVMM limits writes. - assert_eq!(child.write, [work.join("out")]); - assert_eq!( - sources(&mapping), - [ - ("0".to_owned(), true), - ("0/out".to_owned(), false), - ("0/src".to_owned(), true), - ("0/out/cache".to_owned(), false), - ] - ); - - // A read-write root covers everything, so OpenVMM need not limit writes. - let policy = FilesystemPolicy { - readwrite_paths: vec![base.join("work"), base.join("work/src")], - ..FilesystemPolicy::default() - }; - let mapping = plan(&policy).unwrap().unwrap(); - assert!(mapping.children[0].writable && mapping.children[0].write.is_empty()); - assert_eq!( - sources(&mapping), - [("0".to_owned(), false), ("0/src".to_owned(), false)] - ); - - // The read-write duplicate of a read-only path is tightened to read-only, so a - // read-write path inside it is the only writable one. - let policy = FilesystemPolicy { - readonly_paths: vec![base.join("work")], - readwrite_paths: vec![base.join("work"), base.join("work/src")], - ..FilesystemPolicy::default() - }; - let mapping = plan(&policy).unwrap().unwrap(); - assert!(mapping.children[0].writable); - assert_eq!(mapping.children[0].write, [work.join("src")]); - assert_eq!( - sources(&mapping), - [("0".to_owned(), true), ("0/src".to_owned(), false)] - ); - } - - /// Only the guest's bind mount could keep a read-only path inside a read-write directory - /// read-only, so such a path is refused on every host, under any of its names, while the - /// inverse nesting and denied paths inside a read-write directory stay valid. - #[test] - fn read_only_paths_inside_read_write_directories_are_refused() { - let directory = root(); - let base = directory.path(); - fs::create_dir(base.join("work/output")).unwrap(); - let policy = |readonly_paths: &[PathBuf], readwrite_paths: &[PathBuf]| FilesystemPolicy { - readonly_paths: readonly_paths.to_vec(), - readwrite_paths: readwrite_paths.to_vec(), - ..FilesystemPolicy::default() - }; - let refusal = |inner: &Path, outer: &Path| { - format!( - "filesystem.readonlyPaths entry {} lies inside the filesystem.readwritePaths \ - entry {},", - inner.display(), - outer.display() - ) - }; - let work = base.join("work"); - let (src, config) = (work.join("src"), work.join("config.json")); - let secret = src.join("secret"); - // A directory, a file, a path deeper inside, a path that is also listed read-write, and a - // path inside a read-write directory that itself lies inside a read-only one. - for (readonly, readwrite, inner, outer) in [ - (vec![src.clone()], vec![work.clone()], &src, &work), - (vec![config.clone()], vec![work.clone()], &config, &work), - (vec![secret.clone()], vec![work.clone()], &secret, &work), - ( - vec![src.clone()], - vec![work.clone(), src.clone()], - &src, - &work, - ), - ( - vec![work.clone(), secret.clone()], - vec![src.clone()], - &secret, - &src, - ), - ] { - let error = plan(&policy(&readonly, &readwrite)).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported, "{error}"); - assert!( - error.message().starts_with(&refusal(inner, outer)), - "{error}" - ); - } - // Resolved paths decide, so another name of a nested path is refused too: a spelling in - // other letter case on Windows, and one through a symbolic link elsewhere. - #[cfg(windows)] - let alias = base.join("WORK").join("CONFIG.JSON"); - #[cfg(unix)] - let alias = { - std::os::unix::fs::symlink(&work, base.join("work-link")).unwrap(); - base.join("work-link").join("config.json") - }; - let error = plan(&policy( - std::slice::from_ref(&alias), - std::slice::from_ref(&work), - )) - .unwrap_err(); - assert!( - error.message().starts_with(&refusal(&alias, &work)), - "{error}" - ); - - // The inverse nesting is valid: OpenVMM limits the read-only directory's writes to the - // read-write path inside it. - let mapping = plan(&policy(std::slice::from_ref(&work), &[work.join("out")])) - .unwrap() - .unwrap(); - assert_eq!(mapping.children[0].write, [canonical(&work.join("out"))]); - // Containment compares whole components, so work/output lies beside work/out. - let mapping = plan(&policy(&[work.join("output")], &[work.join("out")])) - .unwrap() - .unwrap(); - assert_eq!(mapping.children.len(), 2); - let denied = FilesystemPolicy { - readwrite_paths: vec![work.clone()], - denied_paths: vec![work.join("private")], - ..FilesystemPolicy::default() - }; - assert_eq!(plan(&denied).unwrap().unwrap().children[0].denied.len(), 1); - } - - #[test] - fn denied_paths_belong_to_the_directory_that_contains_them() { - let directory = root(); - let base = directory.path(); - let policy = FilesystemPolicy { - readonly_paths: vec![base.join("work/src"), base.join("projects")], - readwrite_paths: vec![base.join("build-output")], - denied_paths: vec![ - base.join("work/src/secret"), - base.join("projects/private"), - base.join("work/out"), - base.join("elsewhere"), - ], - }; - let mapping = plan(&policy).unwrap().unwrap(); - let denied: Vec> = mapping - .children - .iter() - .map(|child| child.denied.clone()) - .collect(); - assert_eq!( - denied, - [ - Vec::new(), - vec![canonical(&base.join("projects/private"))], - vec![canonical(&base.join("work/src/secret"))], - ] - ); - // Nothing a workload could move is pinned without a read-write parent. - assert!( - mapping - .children - .iter() - .flat_map(|child| &child.denied_identities) - .all(Option::is_none) - ); - assert_eq!( - option_values(&mapping.openvmm_arguments(), "--mount-deny").len(), - 2 - ); - } - - /// Checks the order, duplicates, and containment by whole components that the roots, denied - /// paths, and writable paths share, which needs no host paths. - #[test] - fn outermost_paths_are_sorted_once_without_the_paths_inside_others() { - let paths = ["/b/c", "/a/bc", "/b", "/a/b", "/b/c/d", "/a/b"].map(PathBuf::from); - assert_eq!( - outermost(paths.to_vec()), - ["/a/b", "/a/bc", "/b"].map(PathBuf::from) - ); - } - - #[test] - fn unenforceable_policies_are_rejected() { - let directory = root(); - let base = directory.path(); - let cases = [ - FilesystemPolicy { - readonly_paths: vec![base.join("missing")], - ..FilesystemPolicy::default() - }, - FilesystemPolicy { - readwrite_paths: vec![base.join("work/out")], - denied_paths: vec![base.join("work")], - ..FilesystemPolicy::default() - }, - FilesystemPolicy { - readwrite_paths: vec![base.join("work/out")], - denied_paths: vec![base.join("work/out/not-yet")], - ..FilesystemPolicy::default() - }, - FilesystemPolicy { - readonly_paths: vec![base.join("work/config.json")], - denied_paths: vec![base.join("work")], - ..FilesystemPolicy::default() - }, - // A whole volume is never exported. - FilesystemPolicy { - readonly_paths: vec![PathBuf::from(if cfg!(windows) { r"C:\" } else { "/" })], - ..FilesystemPolicy::default() - }, - ]; - for policy in cases { - assert_eq!( - plan(&policy).unwrap_err().kind(), - ErrorKind::Unsupported, - "{policy:?}" - ); - } - if !cfg!(windows) { - let system = FilesystemPolicy { - readonly_paths: vec![PathBuf::from("/usr")], - ..FilesystemPolicy::default() - }; - assert_eq!(plan(&system).unwrap_err().kind(), ErrorKind::Unsupported); - } - let empty = FilesystemPolicy { - denied_paths: vec![base.join("tools")], - ..FilesystemPolicy::default() - }; - assert_eq!(plan(&empty).unwrap(), None); - } - - #[test] - fn whole_volumes_are_refused() { - // The paths are only inspected, so nothing is created at a volume root. A file directly - // in a volume's root is exported alone, so only a whole volume is refused. - let (volumes, directories): (&[&str], &[&str]) = if cfg!(windows) { - (&[r"C:\", r"D:\"], &[r"C:\work", r"D:\a\b"]) - } else { - (&["/"], &["/work", "/srv/a/b"]) - }; - for volume in volumes { - let reason = volume_root_refusal(Path::new(volume)).unwrap(); - assert!(reason.contains("whole volume"), "{volume}: {reason}"); - } - for directory in directories { - assert_eq!(volume_root_refusal(Path::new(directory)), None); - } - } - - #[test] - fn spellings_of_one_object_share_its_guest_path() { - let directory = root(); - let base = directory.path(); - let upper = if cfg!(windows) { - base.join("WORK").join("SRC") - } else { - base.join("work").join(".").join("src") - }; - let policy = FilesystemPolicy { - readonly_paths: vec![upper], - readwrite_paths: vec![base.join("work").join("src")], - ..FilesystemPolicy::default() - }; - let mapping = plan(&policy).unwrap().unwrap(); - assert_eq!(mapping.binds.len(), 1); - assert!(mapping.binds[0].read_only); - assert!(!mapping.children[0].writable); - assert_eq!( - mapping.binds[0].target, - guest_path(&canonical(&base.join("work").join("src"))).unwrap() - ); - } - - #[cfg(windows)] - #[test] - fn resolved_guest_paths_canonicalize_temp_directory_case_aliases() { - let directory = root(); - let path = directory.path().join("work").join("out"); - let upper = PathBuf::from(path.to_str().unwrap().to_uppercase()); - let expected = guest_path(&canonical(&path)).unwrap(); - assert_ne!(guest_path(&upper), Some(expected.clone())); - assert_eq!(resolve_guest_path(&upper), Some(expected)); - } - - #[cfg(windows)] - #[test] - fn paths_that_only_a_verbatim_path_names_are_refused() { - let directory = root(); - let base = directory.path(); - // A verbatim path keeps a trailing period or space of a name, which Windows drops from a - // path without the prefix: there, `work.` and `work ` would name `work`, and `solo.` - // nothing. - let verbatim = fs::canonicalize(base).unwrap(); - let named = |name: &str| PathBuf::from(format!(r"{}\{name}", verbatim.display())); - for name in ["work.", "work ", "solo.", r"work\src\secret."] { - fs::create_dir(named(name)).unwrap(); - } - for name in ["work.", "work ", "solo."] { - let path = named(name); - let error = canonicalize(&path).unwrap_err(); - assert_eq!(error.kind(), std::io::ErrorKind::Unsupported, "{error}"); - let policy = FilesystemPolicy { - readwrite_paths: vec![path], - ..FilesystemPolicy::default() - }; - let error = plan(&policy).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported); - assert!(error.to_string().contains("is not accessible"), "{error}"); - } - // A denied path that only a verbatim path names would hide another path instead. - let policy = FilesystemPolicy { - readwrite_paths: vec![base.join("work")], - denied_paths: vec![named(r"work\src\secret.")], - ..FilesystemPolicy::default() - }; - let error = plan(&policy).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported); - assert!(error.to_string().contains("cannot be resolved"), "{error}"); - // A verbatim path that names what the path without its prefix names is planned as that - // path. - let policy = FilesystemPolicy { - readwrite_paths: vec![named("work")], - ..FilesystemPolicy::default() - }; - let exported = plan(&policy).unwrap().unwrap(); - assert_eq!(exported.children[0].root, canonical(&base.join("work"))); - } - - #[test] - fn policy_paths_follow_openvmm_rules() { - let directory = root(); - let base = directory.path(); - for name in ["work/out/My Secrets", "work/out/ leading", "my files"] { - fs::create_dir_all(base.join(name)).unwrap(); - } - fs::write(base.join("my files/notes one.txt"), b"notes").unwrap(); - // Names may contain spaces, in a file that a child exposes alone and in a policy path. - let spaced = FilesystemPolicy { - readonly_paths: vec![base.join("my files/notes one.txt")], - readwrite_paths: vec![base.join("work/out")], - denied_paths: vec![base.join("work/out/My Secrets")], - }; - let mapping = plan(&spaced).unwrap().unwrap(); - let arguments = mapping.openvmm_arguments(); - assert!( - option_values(&arguments, "--mount-child")[0].ends_with("notes one.txt,ro,file"), - "{arguments:?}" - ); - assert!(option_values(&arguments, "--mount-deny")[0].ends_with("My Secrets")); - // But they must not begin or end with one. - let leading = FilesystemPolicy { - readwrite_paths: vec![base.join("work/out")], - denied_paths: vec![base.join("work/out/ leading")], - ..FilesystemPolicy::default() - }; - assert_eq!(plan(&leading).unwrap_err().kind(), ErrorKind::Unsupported); - if !cfg!(windows) { - fs::create_dir_all(base.join("work/out/tab\there")).unwrap(); - fs::create_dir_all(base.join("work/out/colon:here")).unwrap(); - for name in ["tab\there", "colon:here"] { - let policy = FilesystemPolicy { - readwrite_paths: vec![base.join("work/out")], - denied_paths: vec![base.join("work/out").join(name)], - ..FilesystemPolicy::default() - }; - assert_eq!( - plan(&policy).unwrap_err().kind(), - ErrorKind::Unsupported, - "{name}" - ); - } - } - } - - #[test] - fn openvmm_limits_are_checked_per_directory() { - let directory = root(); - let base = directory.path(); - let files = |count: usize| -> Vec { - (0..count) - .map(|index| { - let path = base.join("tools").join(format!("tool-{index:03}")); - fs::write(&path, b"x").unwrap(); - path - }) - .collect() - }; - // One exported directory hides at most 128 denied paths, and singles out at most 128 - // read-write paths inside a read-only one. - for policy in [ - FilesystemPolicy { - readonly_paths: vec![base.join("tools")], - denied_paths: files(129), - ..FilesystemPolicy::default() - }, - FilesystemPolicy { - readonly_paths: vec![base.join("tools")], - readwrite_paths: files(129), - ..FilesystemPolicy::default() - }, - ] { - let error = plan(&policy).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported); - assert!(error.message().contains("at most 128"), "{error}"); - } - } - - #[test] - fn swapped_objects_fail_verification() { - let directory = root(); - let base = directory.path(); - // Everything below the read-write work directory is pinned. - let policy = FilesystemPolicy { - readwrite_paths: vec![base.join("work"), base.join("work/src")], - denied_paths: vec![base.join("work/src/secret")], - ..FilesystemPolicy::default() - }; - let mapping = plan(&policy).unwrap().unwrap(); - assert!( - mapping.children[0] - .denied_identities - .iter() - .all(Option::is_some) - ); - mapping.verify().unwrap(); - fs::rename(base.join("work/src/secret"), base.join("work/src/moved")).unwrap(); - fs::create_dir(base.join("work/src/secret")).unwrap(); - assert_eq!(mapping.verify().unwrap_err().kind(), ErrorKind::Other); - fs::remove_dir(base.join("work/src/secret")).unwrap(); - fs::rename(base.join("work/src/moved"), base.join("work/src/secret")).unwrap(); - mapping.verify().unwrap(); - fs::rename(base.join("work/src"), base.join("work/src-moved")).unwrap(); - fs::create_dir_all(base.join("work/src/secret")).unwrap(); - assert_eq!(mapping.verify().unwrap_err().kind(), ErrorKind::Other); - - // Without a read-write parent, nothing a workload could move is pinned, so host edits - // that replace objects do not block the next start. - let read_only = FilesystemPolicy { - readonly_paths: vec![base.join("tools")], - ..FilesystemPolicy::default() - }; - let mapping = plan(&read_only).unwrap().unwrap(); - fs::remove_dir_all(base.join("tools")).unwrap(); - fs::create_dir(base.join("tools")).unwrap(); - mapping.verify().unwrap(); - - // Every child keeps its kind: a directory in place of a file child would grant all of it, - // and a directory child must remain a directory. - let settings = base.join("settings.json"); - fs::write(&settings, b"{}").unwrap(); - let with_file = FilesystemPolicy { - readonly_paths: vec![settings.clone(), base.join("tools")], - ..FilesystemPolicy::default() - }; - let mapping = plan(&with_file).unwrap().unwrap(); - mapping.verify().unwrap(); - fs::remove_file(&settings).unwrap(); - fs::create_dir(&settings).unwrap(); - assert_eq!(mapping.verify().unwrap_err().kind(), ErrorKind::Other); - fs::remove_dir(&settings).unwrap(); - fs::write(&settings, b"{}").unwrap(); - mapping.verify().unwrap(); - fs::remove_dir(base.join("tools")).unwrap(); - fs::write(base.join("tools"), b"").unwrap(); - assert_eq!(mapping.verify().unwrap_err().kind(), ErrorKind::Other); - } - - /// A recorded plan with a read-only bind in a part of a child that the guest may write, which - /// only the guest's bind mount would keep read-only, is refused before a guest runs. - #[test] - fn read_only_binds_that_the_host_cannot_enforce_fail_verification() { - let directory = root(); - let base = directory.path(); - fs::create_dir_all(base.join("work/out/cache")).unwrap(); - let refused = |mut mapping: ExportPlan, source: &str| { - let bind = mapping - .binds - .iter_mut() - .find(|bind| bind.source == source) - .unwrap(); - bind.read_only = true; - let error = mapping.verify().unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported, "{error}"); - assert!( - error.message().contains("read-only inside a part of"), - "{error}" - ); - }; - // A read-only path inside a read-write root, all of which the guest may write. - let writable_root = plan(&FilesystemPolicy { - readwrite_paths: vec![base.join("work"), base.join("work/src")], - ..FilesystemPolicy::default() - }) - .unwrap() - .unwrap(); - writable_root.verify().unwrap(); - refused(writable_root, "src"); - // A read-only path inside a writable path of a read-only root, whose other read-only - // paths stay read-only on the host. - let writable_path = plan(&FilesystemPolicy { - readonly_paths: vec![base.join("work"), base.join("work/src")], - readwrite_paths: vec![base.join("work/out"), base.join("work/out/cache")], - ..FilesystemPolicy::default() - }) - .unwrap() - .unwrap(); - writable_path.verify().unwrap(); - refused(writable_path, "out/cache"); - } - - /// A directory on another volume than the temporary directory, if the host has one: the one - /// that `NVX_TEST_SECOND_VOLUME` names, or the runner's temporary directory - /// on GitHub's Windows runners. Tests create their files below it. Linux's `/dev/shm` does - /// not qualify, because it lies in the guest's reserved `/dev`. - fn second_volume() -> Option { - let candidate = std::env::var_os("NVX_TEST_SECOND_VOLUME") - .or_else(|| std::env::var_os("RUNNER_TEMP")) - .map(PathBuf::from)?; - let temporary = canonical(&std::env::temp_dir()); - let candidate = canonicalize(&candidate).ok()?; - let volume = |path: &Path| platform::file_identity(path).ok().map(|(device, _)| device); - (candidate.is_dir() && volume(&candidate) != volume(&temporary)).then_some(candidate) - } - - #[test] - fn directories_on_other_volumes_are_exported_too() { - let Some(volume) = second_volume() else { - eprintln!("skipped: the host has no second volume"); - return; - }; - let directory = root(); - let other = tempfile::tempdir_in(volume).unwrap(); - fs::create_dir(other.path().join("cache")).unwrap(); - let policy = FilesystemPolicy { - readonly_paths: vec![directory.path().join("tools")], - readwrite_paths: vec![other.path().join("cache")], - ..FilesystemPolicy::default() - }; - let mapping = plan(&policy).unwrap().unwrap(); - assert_eq!(mapping.children.len(), 2); - let modes: Vec = mapping - .children - .iter() - .map(|child| child.writable) - .collect(); - assert!(modes.contains(&true) && modes.contains(&false)); - } - /// Returns a mapping of `path` at the guest path `target`. - fn at<'a>(path: &'a Path, target: &'a str, read_only: bool) -> Mapping<'a> { - Mapping { - name: "mappings", - path, - target: Target::Guest(target), - read_only, - } - } - - #[test] - fn explicit_targets_place_mappings_at_any_free_guest_path() { - let directory = root(); - let base = directory.path(); - let (work, tools) = (base.join("work"), base.join("tools")); - let config = work.join("config.json"); - let mappings = [ - at(&work, "/workspace", false), - at(&tools, "/opt/tools", true), - // A target inside another shows what the other shows there. - at(&config, "/workspace/config.json", false), - // One object may appear at several targets. - at(&work, "/mirror", false), - ]; - let request = ExportRequest::new(&mappings, ReservedTargets::CONTAINER_ROOT); - let mapping = super::plan(&request).unwrap().unwrap(); - let roots: Vec<&Path> = mapping - .children - .iter() - .map(|child| child.root.as_path()) - .collect(); - assert_eq!(roots, [canonical(&tools), canonical(&work)]); - let binds: Vec<(&str, String, bool)> = mapping - .binds - .iter() - .map(|bind| (bind.target.as_str(), bind.guest_source(), bind.read_only)) - .collect(); - assert_eq!( - binds, - [ - ("/mirror", "1".to_owned(), false), - ("/workspace", "1".to_owned(), false), - ("/opt/tools", "0".to_owned(), true), - ("/workspace/config.json", "1/config.json".to_owned(), false), - ] - ); - mapping.check(ReservedTargets::CONTAINER_ROOT).unwrap(); - // A root that keeps its whole tree refuses every mapping, planned or recorded. - let everything = ReservedTargets { - trees: &["/"], - exact: &[], - }; - for error in [ - super::plan(&ExportRequest::new(&mappings, everything)).unwrap_err(), - mapping.check(everything).unwrap_err(), - ] { - assert_eq!(error.kind(), ErrorKind::Unsupported, "{error}"); - assert!( - error.message().ends_with("would cover the guest's /"), - "{error}" - ); - } - // The convention's plans pass the check of their root too. - let convention = plan(&FilesystemPolicy { - readonly_paths: vec![tools.clone()], - readwrite_paths: vec![work.clone(), config.clone()], - ..FilesystemPolicy::default() - }) - .unwrap() - .unwrap(); - convention.check(ReservedTargets::BLOCKLESS_ROOT).unwrap(); - - let refused = |mappings: &[Mapping<'_>], expected: &str| { - let request = ExportRequest::new(mappings, ReservedTargets::CONTAINER_ROOT); - let error = super::plan(&request).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported, "{error}"); - assert!( - error.message().contains(expected), - "{error} lacks {expected}" - ); - }; - refused( - &[at(&work, "/data", true), at(&tools, "/data", true)], - "have the same guest target /data", - ); - refused( - &[at(&work, "/data", true), at(&tools, "/data/tools", true)], - "at /data/tools would hide part of the mappings entry", - ); - // A file shows nothing below it. - refused( - &[at(&config, "/data", true), at(&tools, "/data/tools", true)], - "would hide part of", - ); - let long = format!("/{}", "x".repeat(4095)); - for (target, problem) in [ - ("data", "is not absolute"), - ("/", "is the guest's root"), - ("/a//b", "has an empty, `.`, or `..` component"), - ("/a/./b", "has an empty, `.`, or `..` component"), - ("/a/../b", "has an empty, `.`, or `..` component"), - ("/a/", "has an empty, `.`, or `..` component"), - ("/a\0b", "contains a NUL character"), - (long.as_str(), "is longer than a guest path may be"), - ] { - refused(&[at(&work, target, true)], problem); - } - refused( - &[at(&work, "/etc", true)], - "has the guest target /etc, which would cover the guest's /etc", - ); - refused( - &[at(&work, "/proc/self", true)], - "would cover the guest's /proc", - ); - refused( - &[at(&config, "/etc/resolv.conf", true)], - "has the guest target /etc/resolv.conf, which would cover the guest's /etc/resolv.conf", - ); - // The same object read-only and read-write at two targets would let the guest write it. - refused( - &[at(&work, "/a", false), at(&work, "/b", true)], - "names the object of the mappings entry", - ); - // A target inside another must spell, name for name, the host path below the other's - // object, so a backslash in a guest name, which a Windows host reads as a separator, and a - // guest name that a Windows host reads as a root of its own show nothing there. - let host = work.join("src").join("host"); - fs::create_dir_all(&host).unwrap(); - refused( - &[ - at(&work, "/workspace", true), - at(&host, r"/workspace/src\host", true), - ], - "would hide part of", - ); - let resolved = canonical(&work); - let text = resolved.to_str().unwrap(); - let rooted = text - .split_once(':') - .map_or(text, |(_, rest)| rest) - .replace('/', r"\"); - let inside = format!("/opt/tools/{rooted}"); - refused( - &[at(&tools, "/opt/tools", true), at(&work, &inside, true)], - "would hide part of", - ); - } - - #[test] - fn reserved_targets_depend_on_the_root() { - let container = ReservedTargets::CONTAINER_ROOT; - for (target, reserved) in [ - ("/", "/"), - ("/proc", "/proc"), - ("/proc/self", "/proc"), - ("/sys/fs", "/sys"), - ("/dev/shm", "/dev"), - ("/.nvx-agent/tools", "/.nvx-agent"), - ("/etc", "/etc"), - ("/etc/machine-id", "/etc/machine-id"), - ("/etc/resolv.conf", "/etc/resolv.conf"), - ("/run", "/run"), - // The ancestors of the runtime's trees are reserved, as exact paths here. - ("/run/nvx", "/run/nvx"), - ("/run/nvx/time/state", "/run/nvx/time"), - ] { - assert_eq!(container.reservation(target), Some(reserved), "{target}"); - } - for target in [ - "/etc/app", - "/etc/resolvconf", - "/run/secrets", - "/run/nvx/other", - "/usr/local", - "/workspace", - "/procfs", - ] { - assert_eq!(container.reservation(target), None, "{target}"); - } - let blockless = ReservedTargets::BLOCKLESS_ROOT; - for (target, reserved) in [ - ("/", "/"), - ("/usr/local", "/usr"), - ("/run", "/run"), - ("/etc/app", "/etc"), - ("/init", "/init"), - ] { - assert_eq!(blockless.reservation(target), Some(reserved), "{target}"); - } - for target in ["/workspace", "/mnt/c/work", "/home/me", "/initrd", "/opt"] { - assert_eq!(blockless.reservation(target), None, "{target}"); - } - // A mapping at an ancestor of a reserved tree would shadow it. - let nested = ReservedTargets { - trees: &["/srv/runtime/state"], - exact: &[], - }; - assert_eq!(nested.reservation("/srv"), Some("/srv/runtime/state")); - assert_eq!( - nested.reservation("/srv/runtime"), - Some("/srv/runtime/state") - ); - assert_eq!(nested.reservation("/srv/other"), None); - // A root that keeps its whole tree takes no mapping. - let everything = ReservedTargets { - trees: &["/"], - exact: &[], - }; - for target in ["/", "/workspace", "/srv/runtime/state"] { - assert_eq!(everything.reservation(target), Some("/"), "{target}"); - } - } - - #[test] - fn policy_paths_hide_show_and_narrow_writes() { - let directory = root(); - let base = directory.path(); - fs::create_dir_all(base.join("work/src/secret/public")).unwrap(); - fs::create_dir_all(base.join("work/out/cache")).unwrap(); - let work = base.join("work"); - let paths = - |names: &[&str]| -> Vec { names.iter().map(|name| work.join(name)).collect() }; - let request = |mappings: &[Mapping<'_>], - denied: &[PathBuf], - allowed: &[PathBuf], - write: &[PathBuf]| { - let request = ExportRequest { - denied: PathList { - name: "denied", - paths: denied, - }, - allowed: PathList { - name: "allowed", - paths: allowed, - }, - write: PathList { - name: "write", - paths: write, - }, - ..ExportRequest::new(mappings, ReservedTargets::CONTAINER_ROOT) - }; - super::plan(&request) - }; - let whole = [at(&work, "/workspace", false)]; - let (denied, allowed, write) = ( - paths(&["src/secret"]), - paths(&["src/secret/public"]), - paths(&["out"]), - ); - let mapping = request(&whole, &denied, &allowed, &write).unwrap().unwrap(); - let child = &mapping.children[0]; - let resolved = canonical(&work); - assert!(child.writable); - assert_eq!(child.denied, [resolved.join("src").join("secret")]); - assert_eq!( - child.allowed, - [resolved.join("src").join("secret").join("public")] - ); - assert_eq!(child.write, [resolved.join("out")]); - // Writes are narrowed to `out`, so nothing outside it can move. - assert_eq!(child.denied_identities, [None]); - let arguments = mapping.openvmm_arguments(); - let text = |path: PathBuf| vec![display_path(&path)]; - assert_eq!( - option_values(&arguments, "--mount-deny"), - text(resolved.join("src").join("secret")) - ); - assert_eq!( - option_values(&arguments, "--mount-allow"), - text(resolved.join("src").join("secret").join("public")) - ); - assert_eq!( - option_values(&arguments, "--mount-write"), - text(resolved.join("out")) - ); - assert!(!arguments.contains(&"--mount-owner".to_owned())); - mapping.check(ReservedTargets::CONTAINER_ROOT).unwrap(); - - // A denied root shows only its allowed paths. - let mapping = request(&whole, std::slice::from_ref(&work), &paths(&["src"]), &[]) - .unwrap() - .unwrap(); - assert_eq!(mapping.children[0].denied, std::slice::from_ref(&resolved)); - assert_eq!(mapping.children[0].allowed, [resolved.join("src")]); - let error = request(&whole, std::slice::from_ref(&work), &[], &[]).unwrap_err(); - assert!( - error.message().contains("contains the mapped path"), - "{error}" - ); - - // A read-only path may lie in the part of a narrowed directory that stays read-only, and - // the directory may be mapped read-only elsewhere too. - let src = work.join("src"); - let parts = [ - at(&work, "/workspace", false), - at(&src, "/src", true), - at(&work, "/view", true), - ]; - request(&parts, &[], &[], &write).unwrap().unwrap(); - - let refused = |mappings: &[Mapping<'_>], - denied: &[PathBuf], - allowed: &[PathBuf], - write: &[PathBuf], - expected: &str| { - let error = request(mappings, denied, allowed, write).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported, "{error}"); - assert!( - error.message().contains(expected), - "{error} lacks {expected}" - ); - }; - refused( - &whole, - &[], - &paths(&["src"]), - &[], - "lies inside no denied entry", - ); - refused( - &whole, - std::slice::from_ref(&work), - std::slice::from_ref(&work), - &[], - "is an exported path itself", - ); - refused( - &whole, - &paths(&["src"]), - &paths(&["src/secret", "src/secret/public"]), - &[], - "which shows it already", - ); - refused( - &whole, - &paths(&["src"]), - &paths(&["src"]), - &[], - "is also a denied entry", - ); - refused( - &[at(&work, "/workspace", true)], - &[], - &[], - &write, - "lies in no directory that is mapped read-write as a whole", - ); - refused( - &whole, - &[], - &[], - std::slice::from_ref(&work), - "lies in no directory that is mapped read-write as a whole", - ); - refused( - &whole, - &paths(&["src"]), - &[], - &paths(&["src/secret"]), - "which hides it", - ); - let cache = work.join("out/cache"); - refused( - &[at(&work, "/workspace", false), at(&cache, "/cache", true)], - &[], - &[], - &write, - "lies inside the write entry", - ); - refused( - &[at(&work, "/workspace", false), at(&work, "/view", true)], - &[], - &[], - &[], - "names the object of the mappings entry", - ); - } - - #[test] - fn narrowed_writes_pin_only_what_lies_in_writable_paths() { - let directory = root(); - let base = directory.path(); - fs::create_dir_all(base.join("work/out/cache")).unwrap(); - fs::create_dir_all(base.join("work/out/private")).unwrap(); - let work = base.join("work"); - let (cache, src) = (work.join("out/cache"), work.join("src")); - let mappings = [ - at(&work, "/workspace", false), - at(&cache, "/cache", false), - at(&src, "/src", true), - ]; - let denied = [work.join("out/private"), work.join("src/secret")]; - let write = [work.join("out")]; - let request = ExportRequest { - denied: PathList { - name: "denied", - paths: &denied, - }, - write: PathList { - name: "write", - paths: &write, - }, - ..ExportRequest::new(&mappings, ReservedTargets::CONTAINER_ROOT) - }; - let mapping = super::plan(&request).unwrap().unwrap(); - let child = &mapping.children[0]; - assert_eq!(child.write, [canonical(&work).join("out")]); - let pinned: Vec = child - .denied_identities - .iter() - .map(Option::is_some) - .collect(); - assert_eq!(pinned, [true, false]); - let pinned: Vec<(&str, bool)> = mapping - .binds - .iter() - .map(|bind| (bind.target.as_str(), bind.identity.is_some())) - .collect(); - assert_eq!( - pinned, - [("/cache", true), ("/src", false), ("/workspace", false)] - ); - // A record keeps every pin of the plan, and narrows writes only to paths that keep its - // read-write mappings writable. - mapping.check(ReservedTargets::CONTAINER_ROOT).unwrap(); - let refused = |change: fn(&mut ExportPlan), expected: &str| { - let mut changed = mapping.clone(); - change(&mut changed); - let error = changed.check(ReservedTargets::CONTAINER_ROOT).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported, "{error}"); - assert!(error.message().contains(expected), "{error}"); - }; - refused( - |changed| changed.children[0].denied_identities[0] = None, - "does not pin the denied path", - ); - refused( - |changed| changed.binds[0].identity = None, - "does not pin the object that it maps at /cache", - ); - refused( - |changed| changed.children[0].write = vec![changed.children[0].root.join("other")], - "which it maps read-write, stays read-only", - ); - // Without write paths, the guest could move `src/secret` too. - refused( - |changed| changed.children[0].write.clear(), - "does not pin the denied path", - ); - mapping.verify().unwrap(); - fs::rename(&cache, work.join("out/moved")).unwrap(); - fs::create_dir(&cache).unwrap(); - assert_eq!(mapping.verify().unwrap_err().kind(), ErrorKind::Other); - } - - #[test] - fn the_caller_owner_reaches_openvmm_on_linux_hosts() { - let directory = root(); - let work = directory.path().join("work"); - let mappings = [at(&work, "/workspace", false)]; - let request = ExportRequest { - owner: Owner::Caller, - ..ExportRequest::new(&mappings, ReservedTargets::CONTAINER_ROOT) - }; - if cfg!(windows) { - let error = super::plan(&request).unwrap_err(); - assert!(error.message().contains("requires a Linux host"), "{error}"); - return; - } - #[cfg(unix)] - { - use std::os::unix::fs::MetadataExt; - - let owned = fs::metadata(&work).unwrap(); - if owned.uid() != 0 && owned.gid() != 0 { - let mapping = super::plan(&request).unwrap().unwrap(); - assert_eq!(mapping.owner, Owner::Caller); - let arguments = mapping.openvmm_arguments(); - assert_eq!( - arguments[arguments.len() - 2..], - ["--mount-owner", "caller"] - ); - mapping.verify().unwrap(); - let record = serde_json::to_value(&mapping).unwrap(); - assert_eq!(record["owner"], "caller"); - assert_eq!( - serde_json::from_value::(record).unwrap(), - mapping - ); - } - // Guest root is squashed to the owner, which must not be root. - let system = Path::new("/usr"); - if fs::metadata(system).is_ok_and(|metadata| metadata.uid() == 0) { - let mappings = [at(system, "/srv/system", true)]; - let request = ExportRequest { - owner: Owner::Caller, - ..ExportRequest::new(&mappings, ReservedTargets::CONTAINER_ROOT) - }; - let error = super::plan(&request).unwrap_err(); - assert!( - error - .message() - .contains("must not be owned by UID 0 or GID 0"), - "{error}" - ); - } - } - // The default owner is left out of the record. - let vmm = super::plan(&ExportRequest::new( - &mappings, - ReservedTargets::CONTAINER_ROOT, - )) - .unwrap() - .unwrap(); - assert!(serde_json::to_value(&vmm).unwrap().get("owner").is_none()); - } - - #[test] - fn read_write_mappings_to_which_writes_are_narrowed_stay_visible() { - let directory = root(); - let base = directory.path(); - fs::create_dir_all(base.join("tree/private/public")).unwrap(); - fs::create_dir_all(base.join("tree/out")).unwrap(); - let (tree, private) = (base.join("tree"), base.join("tree/private")); - let denied = [private.clone()]; - let allowed = [private.join("public")]; - let request = |mappings: &[Mapping<'_>], write: &[PathBuf]| { - super::plan(&ExportRequest { - denied: PathList { - name: "denied", - paths: &denied, - }, - allowed: PathList { - name: "allowed", - paths: &allowed, - }, - write: PathList { - name: "write", - paths: write, - }, - ..ExportRequest::new(mappings, ReservedTargets::CONTAINER_ROOT) - }) - }; - // OpenVMM narrows the writes of the tree to the read-write private, which it cannot hide: - // in a read-only tree, and in a read-write tree whose writes are narrowed. - let out = [tree.join("out")]; - for (mappings, write) in [ - ( - [at(&tree, "/tree", true), at(&private, "/private", false)], - &[][..], - ), - ( - [at(&tree, "/tree", false), at(&private, "/private", false)], - &out[..], - ), - ] { - let error = request(&mappings, write).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported, "{error}"); - assert!( - error - .message() - .contains("OpenVMM cannot make a hidden path writable"), - "{error}" - ); - } - // In a tree that is writable as a whole, the denied private shows its allowed paths. - let mapping = request( - &[at(&tree, "/tree", false), at(&private, "/private", false)], - &[], - ) - .unwrap() - .unwrap(); - let child = &mapping.children[0]; - assert!(child.writable && child.write.is_empty()); - assert_eq!(child.denied, [canonical(&private)]); - assert_eq!(child.allowed, [canonical(&private).join("public")]); - } - - #[test] - fn denied_mapped_directories_inside_denied_ones_show_their_allowed_paths() { - let directory = root(); - let base = directory.path(); - fs::create_dir_all(base.join("tree/private/public")).unwrap(); - fs::create_dir_all(base.join("tree/other")).unwrap(); - let (tree, private) = (base.join("tree"), base.join("tree/private")); - let denied = [tree.clone(), private.clone()]; - let request = |mappings: &[Mapping<'_>], allowed: &[PathBuf]| { - super::plan(&ExportRequest { - denied: PathList { - name: "denied", - paths: &denied, - }, - allowed: PathList { - name: "allowed", - paths: allowed, - }, - ..ExportRequest::new(mappings, ReservedTargets::CONTAINER_ROOT) - }) - }; - let both = [at(&tree, "/tree", false), at(&private, "/private", false)]; - // Each mapped directory is denied itself and shows the allowed path inside it; OpenVMM - // needs only the outer denial, which hides the inner one already. - let mapping = request(&both, &[private.join("public")]).unwrap().unwrap(); - let child = &mapping.children[0]; - assert_eq!(child.denied, [canonical(&tree)]); - assert_eq!(child.allowed, [canonical(&private).join("public")]); - mapping.check(ReservedTargets::CONTAINER_ROOT).unwrap(); - // OpenVMM receives each exported directory's policy alone, so one that a denied directory - // outside it contains keeps its own denial. - let mapping = request( - &[at(&private, "/private", false)], - &[private.join("public")], - ) - .unwrap() - .unwrap(); - let child = &mapping.children[0]; - assert_eq!(child.root, canonical(&private)); - assert_eq!(child.denied, [canonical(&private)]); - assert_eq!(child.allowed, [canonical(&private).join("public")]); - mapping.check(ReservedTargets::CONTAINER_ROOT).unwrap(); - // A mapped directory whose denial shows nothing again inside it stays hidden. - let error = request(&both, &[tree.join("other")]).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported, "{error}"); - assert!( - error.message().contains("contains the mapped path"), - "{error}" - ); - } - - #[test] - fn recorded_plans_hold_no_more_policy_than_one_aggregate_accepts() { - // Each child's paths of each kind stay within OpenVMM's limits for one directory. - let child = |index: usize| { - let root = std::env::temp_dir().join(format!("nvx-policy-{index}")); - let long = |kind: &str, path: usize| format!("{kind}-{path}-{}", "x".repeat(3990)); - let paths = |kind: &str| -> Vec { - (0..4).map(|path| root.join(long(kind, path))).collect() - }; - // OpenVMM shows paths again only inside a path that it hides. - let hidden = root.join(long("denied", 0)); - ExportChild { - root: root.clone(), - file: false, - writable: true, - denied: paths("denied"), - denied_identities: vec![None; 4], - allowed: (0..4) - .map(|path| hidden.join(format!("allowed-{path}"))) - .collect(), - write: paths("write"), - } - }; - let recorded = |count: usize| ExportPlan { - children: (0..count).map(child).collect(), - binds: (0..count) - .map(|index| Bind { - child: index, - source: String::new(), - target: format!("/c{index}"), - read_only: false, - identity: None, - }) - .collect(), - owner: Owner::Vmm, - }; - recorded(2).check(ReservedTargets::CONTAINER_ROOT).unwrap(); - let error = recorded(3) - .check(ReservedTargets::CONTAINER_ROOT) - .unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported, "{error}"); - assert!( - error.message().contains("exceed its 131072-byte limit"), - "{error}" - ); - } - - #[cfg(target_os = "linux")] - #[test] - fn mapped_paths_resolve_to_utf8_paths() { - use std::ffi::OsStr; - use std::os::unix::ffi::OsStrExt; - - let directory = root(); - let base = directory.path(); - let share = base.join(OsStr::from_bytes(b"share-\xff")); - fs::create_dir(&share).unwrap(); - // A link of a valid name resolves to the same path, which OpenVMM's options could not - // name. - let link = base.join("link"); - std::os::unix::fs::symlink(&share, &link).unwrap(); - for path in [&share, &link] { - let mappings = [at(path, "/workspace", false)]; - let error = super::plan(&ExportRequest::new( - &mappings, - ReservedTargets::CONTAINER_ROOT, - )) - .unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported, "{error}"); - assert!( - error - .message() - .contains("resolves to a path that is not valid UTF-8"), - "{error}" - ); - } - } -} diff --git a/nvx-rs/src/host/filesystem/guest_path.rs b/nvx-rs/src/host/filesystem/guest_path.rs new file mode 100644 index 000000000..6f92b54f0 --- /dev/null +++ b/nvx-rs/src/host/filesystem/guest_path.rs @@ -0,0 +1,82 @@ +//! Translation of host paths and aggregate sources into guest paths. + +use std::path::{Component, Path, Prefix}; + +use super::host_path::canonicalize; + +/// Joins an aggregate child index and child-relative source into a guest export path. +pub(super) fn guest_source(child: usize, source: &str) -> String { + if source.is_empty() { + child.to_string() + } else { + format!("{child}/{source}") + } +} + +/// Returns the guest path at which a sandbox exposes the host path `path`, or `None` if it +/// has no guest equivalent (relative paths, UNC paths, or paths that are not valid UTF-8). +/// +/// A Windows path `C:\work\src` maps to `/mnt/c/work/src`, and an absolute Linux path maps to +/// itself. `.` components are dropped; `..` components are rejected. +pub fn guest_path(path: &Path) -> Option { + let mut components = path.components().peekable(); + let mut guest = match components.peek()? { + Component::Prefix(prefix) => { + let drive = match prefix.kind() { + Prefix::Disk(drive) | Prefix::VerbatimDisk(drive) => drive, + _ => return None, + }; + components.next(); + if !matches!(components.next()?, Component::RootDir) { + return None; + } + format!("/mnt/{}", char::from(drive).to_ascii_lowercase()) + } + Component::RootDir if !cfg!(windows) => { + components.next(); + String::new() + } + _ => return None, + }; + for component in components { + match component { + Component::Normal(name) => { + guest.push('/'); + guest.push_str(name.to_str()?); + } + Component::CurDir => {} + _ => return None, + } + } + if guest.is_empty() { + guest.push('/'); + } + Some(guest) +} + +/// Returns the guest path of an existing host path the way mappings derive it: from the +/// resolved path, so links, letter case, and short names do not matter. A path that does not +/// exist is translated as written. +/// +/// Use it to turn a host working directory into a `process.cwd` inside a mapped path. +pub fn resolve_guest_path(path: &Path) -> Option { + match canonicalize(path) { + Ok(canonical) => guest_path(&canonical), + Err(_) => guest_path(path), + } +} + +/// Returns whether the guest path `path` is `directory` or one of its descendants, comparing +/// whole components: `/run/nvx` is within `/run`, but `/runner` is not, and every absolute path is +/// within the root, `/`. +pub(crate) fn within(path: &str, directory: &str) -> bool { + path == directory + || path + .strip_prefix(directory) + .is_some_and(|rest| directory.ends_with('/') || rest.starts_with('/')) +} + +/// Counts the nonempty components of an absolute guest path. +pub(super) fn depth(path: &str) -> usize { + path.split('/').filter(|part| !part.is_empty()).count() +} diff --git a/nvx-rs/src/host/filesystem/host_path.rs b/nvx-rs/src/host/filesystem/host_path.rs new file mode 100644 index 000000000..e77f8f820 --- /dev/null +++ b/nvx-rs/src/host/filesystem/host_path.rs @@ -0,0 +1,116 @@ +//! Canonical host paths, file identities, and the text of relative host paths. + +use std::fs; +use std::path::{Path, PathBuf}; + +use super::FileIdentity; +use crate::error::{Error, Result}; +use crate::host::platform; + +#[cfg(windows)] +/// Returns whether Windows marks `metadata` as a reparse point. +pub(crate) fn is_reparse_point(metadata: &fs::Metadata) -> bool { + use std::os::windows::fs::MetadataExt; + metadata.file_attributes() & 0x400 != 0 +} + +#[cfg(not(windows))] +/// Reports no reparse points on hosts whose metadata has no such Windows attribute. +pub(crate) fn is_reparse_point(_metadata: &fs::Metadata) -> bool { + false +} + +/// Returns the stable device-and-file identity of `path` for rename detection. +pub(super) fn identity(path: &Path) -> Result { + platform::file_identity(path) + .map(|(device, index)| FileIdentity { device, index }) + .map_err(|error| { + Error::unsupported(format!("cannot identify {}", path.display())).with_source(error) + }) +} + +/// Path of `path` relative to `root`, `/`-separated; empty for the root itself. +pub(super) fn relative_text(root: &Path, path: &Path) -> Option { + let relative = path.strip_prefix(root).ok()?; + let mut parts = Vec::new(); + for component in relative.components() { + parts.push(component.as_os_str().to_str()?); + } + Some(parts.join("/")) +} + +/// Resolves links and returns a path without the Windows verbatim prefix, which must still name +/// the object that the resolved path names. +/// +/// Windows reads a path without the prefix differently: it drops a trailing period or space of a +/// name, among other things. A path that resolves to itself again names the same object; one that +/// only the verbatim prefix keeps, such as that of a directory whose name ends in a period, would +/// name another object, or none, so it is refused. +pub(crate) fn canonicalize(path: &Path) -> std::io::Result { + let canonical = fs::canonicalize(path)?; + let Some(stripped) = strip_verbatim(&canonical) else { + return Err(std::io::Error::new( + std::io::ErrorKind::Unsupported, + "network paths are unsupported", + )); + }; + if stripped != canonical && !fs::canonicalize(&stripped).is_ok_and(|again| again == canonical) { + return Err(std::io::Error::new( + std::io::ErrorKind::Unsupported, + "only a verbatim path names it, as with a name that ends in a period or a space", + )); + } + Ok(stripped) +} + +/// Canonicalizes the deepest existing ancestor and appends the rest. Returns whether the +/// complete path exists. +pub(super) fn canonicalize_lenient(path: &Path) -> std::io::Result<(PathBuf, bool)> { + let mut existing = path.to_path_buf(); + let mut missing = Vec::new(); + loop { + match canonicalize(&existing) { + Ok(canonical) => { + let exists = missing.is_empty(); + let mut resolved = canonical; + for component in missing.iter().rev() { + resolved.push(component); + } + return Ok((resolved, exists)); + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => { + let Some(name) = existing.file_name().map(ToOwned::to_owned) else { + return Err(error); + }; + missing.push(name); + if !existing.pop() { + return Err(error); + } + } + Err(error) => return Err(error), + } + } +} + +#[cfg(windows)] +/// Removes a local Windows verbatim prefix and rejects UNC paths. +fn strip_verbatim(path: &Path) -> Option { + let text = path.to_str()?; + match text.strip_prefix(r"\\?\") { + Some(rest) if rest.starts_with("UNC\\") => None, + Some(rest) => Some(PathBuf::from(rest)), + None if text.starts_with(r"\\") => None, + None => Some(path.to_path_buf()), + } +} + +#[cfg(not(windows))] +/// Returns the unchanged path on hosts without Windows verbatim prefixes. +fn strip_verbatim(path: &Path) -> Option { + Some(path.to_path_buf()) +} + +/// Returns a lossily decoded path for user-facing diagnostics. +pub(super) fn display_path(path: &Path) -> String { + path.to_string_lossy().into_owned() +} diff --git a/nvx-rs/src/host/filesystem/limits.rs b/nvx-rs/src/host/filesystem/limits.rs new file mode 100644 index 000000000..0eb273e69 --- /dev/null +++ b/nvx-rs/src/host/filesystem/limits.rs @@ -0,0 +1,128 @@ +//! OpenVMM's limits and host checks for exported roots and access-policy paths. + +use std::fs; +use std::path::{Component, Path, PathBuf}; + +use super::host_path::{is_reparse_point, relative_text}; +use super::{MAX_POLICY_BYTES, MAX_POLICY_PATH_BYTES, MAX_POLICY_PATHS}; +use crate::error::{Error, Result}; +#[cfg(unix)] +use crate::host::platform; + +/// Applies OpenVMM's rules for exported directories, which it reaches without links. +pub(super) fn check_root(root: &Path) -> Result<()> { + let mut current = PathBuf::new(); + for component in root.components() { + current.push(component.as_os_str()); + if !matches!(component, Component::Normal(_)) { + continue; + } + let metadata = fs::symlink_metadata(¤t).map_err(|error| { + Error::unsupported(format!("cannot inspect {}", current.display())).with_source(error) + })?; + if metadata.file_type().is_symlink() || is_reparse_point(&metadata) { + return Err(Error::unsupported(format!( + "OpenVMM cannot export {}, because {} is a link or reparse point", + root.display(), + current.display() + ))); + } + } + Ok(()) +} + +/// Applies OpenVMM's rules for the paths that it is to `action` in the exported directory `root`, +/// so a policy that OpenVMM would refuse fails at provision rather than at every start: those of +/// [`policy_limits`], and on the host, no link or reparse point below `root` and no other file +/// system. Returns their combined length. +pub(super) fn check_policy_paths(root: &Path, paths: &[PathBuf], action: &str) -> Result { + let unsupported = |message: String| Err(Error::unsupported(message)); + let total = policy_limits(root, paths, action).map_err(Error::unsupported)?; + for path in paths { + // `policy_limits` read each path as text below `root` already. + let relative = relative_text(root, path).unwrap_or_default(); + // The exported directory itself, which a denied root names, was checked as a root. + let components = relative + .split('/') + .filter(|component| !component.is_empty()); + let mut current = root.to_path_buf(); + for component in components { + current.push(component); + let metadata = fs::symlink_metadata(¤t).map_err(|error| { + Error::unsupported(format!("cannot inspect {}", current.display())) + .with_source(error) + })?; + if metadata.file_type().is_symlink() || is_reparse_point(&metadata) { + return unsupported(format!( + "OpenVMM cannot {action} {}, because {} is a link or reparse point", + path.display(), + current.display() + )); + } + } + #[cfg(unix)] + if platform::file_identity(path).ok().map(|(device, _)| device) + != platform::file_identity(root).ok().map(|(device, _)| device) + { + return unsupported(format!( + "OpenVMM cannot {action} {}, which lies on another file system than {}", + path.display(), + root.display() + )); + } + } + Ok(total) +} + +/// Applies those of OpenVMM's rules for the paths that it is to `action` in the exported directory +/// `root` that read nothing from the host: at most [`MAX_POLICY_PATHS`] paths, each valid UTF-8 +/// below `root`, of at most [`MAX_POLICY_PATH_BYTES`] bytes there, without colons, backslashes, +/// whitespace other than spaces, or a name that begins or ends with a space, and of at most +/// [`MAX_POLICY_BYTES`] bytes together. Returns their combined length, or why OpenVMM would refuse +/// them. +pub(super) fn policy_limits( + root: &Path, + paths: &[PathBuf], + action: &str, +) -> std::result::Result { + let count = paths.len(); + if count > MAX_POLICY_PATHS { + return Err(format!( + "OpenVMM can {action} at most {MAX_POLICY_PATHS} paths in one exported directory, \ + but {} needs {count}", + root.display() + )); + } + let mut total = 0; + for path in paths { + let Some(relative) = relative_text(root, path) else { + return Err(format!("{} is not valid UTF-8", path.display())); + }; + // A name may contain spaces, but not begin or end with one. + if relative.len() > MAX_POLICY_PATH_BYTES + || relative.chars().any(|character| { + (character.is_whitespace() && character != ' ') || matches!(character, ':' | '\\') + }) + || relative + .split('/') + .any(|name| name.starts_with(' ') || name.ends_with(' ')) + { + return Err(format!( + "OpenVMM cannot {action} {}: below the exported directory {}, a path must not \ + contain colons, backslashes, or whitespace other than spaces, and no name may \ + begin or end with a space", + path.display(), + root.display() + )); + } + total += relative.len(); + } + if total > MAX_POLICY_BYTES { + return Err(format!( + "the paths that OpenVMM is to {action} in {} exceed its {MAX_POLICY_BYTES}-byte \ + limit", + root.display() + )); + } + Ok(total) +} diff --git a/nvx-rs/src/host/filesystem/mod.rs b/nvx-rs/src/host/filesystem/mod.rs new file mode 100644 index 000000000..7ae455e3a --- /dev/null +++ b/nvx-rs/src/host/filesystem/mod.rs @@ -0,0 +1,807 @@ +//! Host paths exposed to the guest through OpenVMM's aggregate virtio-fs export, as the one +//! planner of every NVX caller plans them, for the blockless root and for the container root of a +//! layered sandbox alike. +//! +//! OpenVMM offers one virtio-fs device. [`plan`] attaches it as an aggregate: a synthetic, +//! read-only root that lists one child per exported host directory or file, named by its index, +//! which the guest mounts at a directory that only the guest's root can enter. The guest then +//! bind-mounts each mapped path at its target, read-only or read-write, in the order of the plan's +//! [`Bind`]s: from the mapping table that it receives over the control channel, or from boot +//! tokens that a caller renders from the same binds. +//! +//! An [`ExportRequest`] names the mapped host paths, each read-only or read-write at its guest +//! target; the host paths that OpenVMM hides inside them, and the paths inside those that stay +//! visible; the paths to which the writes of a read-write directory are narrowed; the host +//! identity of the file operations; and the guest targets that the guest's root keeps for itself. +//! +//! - A caller gives each target, or lets MXC's convention for Linux guests derive it from the +//! resolved host path: a Windows path `C:\work\src` appears at `/mnt/c/work/src`, and a Linux +//! path at the same path ([`fn@guest_path`]). +//! - The reserved targets depend on the root, and the caller supplies them ([`ReservedTargets`]): +//! the blockless root keeps the guest's system trees, and the container root of a layered +//! sandbox keeps the trees of its runtime. +//! +//! The children are the outermost mapped directories and the mapped files outside them. OpenVMM +//! exports each such file alone, without its directory, so mapped paths may lie anywhere, on any +//! volume, except that a whole volume cannot be mapped. A child is read-write only if it holds a +//! read-write mapping, and OpenVMM then limits writes to the child's writable parts, so the host +//! enforces each mapping's access itself, and guest mount flags are never the boundary. OpenVMM +//! cannot keep a path read-only inside a writable part, so a read-only path there is refused on +//! every host, while a read-write path inside a read-only directory is not. OpenVMM also hides the +//! denied paths, except for the allowed paths inside them. +//! +//! The plan records the identities of the mapped and denied paths that a workload could move, +//! those inside a writable part, and [`ExportPlan::verify`] checks them again before each start. +//! A policy that would need more children than OpenVMM offers is refused rather than served +//! through the directories of its files, which would grant more than the files. + +mod guest_path; +mod host_path; +mod limits; +mod policy; +mod verify; + +use std::fs; +use std::path::{Component, Path, PathBuf, Prefix}; + +use serde::{Deserialize, Serialize}; + +use crate::error::{Error, Result}; +use crate::protocol::MAX_HOST_MAPPINGS; +use crate::protocol::export::EXPORT_DIRECTORY; +use crate::protocol::share::{RESERVED_EXACT_MOUNT_TARGETS, RESERVED_MOUNT_TARGETS}; + +pub(crate) use self::guest_path::within; +use self::guest_path::{depth, guest_source}; +pub use self::guest_path::{guest_path, resolve_guest_path}; +pub(crate) use self::host_path::{canonicalize, is_reparse_point}; +use self::host_path::{identity, relative_text}; +use self::limits::{check_policy_paths, check_root}; +use self::policy::{ + Writable, allowed_paths, canonicals, check_allowed_paths, check_read_only, check_write_paths, + denied_paths, distinct_roots, nearest_rule, needed_denied_paths, writable_part, write_paths, +}; + +/// Guest directory where the guest's init mounts the export, which the export's `virtfs_dir=` +/// boot token names, and guests check: [`EXPORT_DIRECTORY`]. Only the guest's root can enter its +/// parent. +pub const GUEST_EXPORT: &str = EXPORT_DIRECTORY; + +/// Largest number of host directories and files that OpenVMM exports through one aggregate. +pub const MAX_CHILDREN: usize = 256; +/// Largest number of hidden, allowed, or writable paths that OpenVMM accepts in one exported +/// directory. +const MAX_POLICY_PATHS: usize = 128; +/// Largest combined UTF-8 byte length of one exported directory's paths of one kind. +const MAX_POLICY_BYTES: usize = 16 * 1024; +/// Largest length of one such path. +const MAX_POLICY_PATH_BYTES: usize = 4096; +/// Largest combined UTF-8 byte length of all policies in one aggregate export. +const MAX_AGGREGATE_POLICY_BYTES: usize = 128 * 1024; +/// Longest guest path, without the terminating NUL. +const MAX_GUEST_PATH: usize = 4095; + +/// Guest directories that a mapping must not cover in the blockless root, because the guest's own +/// system lives there. +pub const RESERVED_TARGETS: [&str; 13] = [ + "/bin", "/boot", "/dev", "/etc", "/lib", "/proc", "/root", "/run", "/sbin", "/sys", "/usr", + "/var", "/init", +]; + +/// Why a request for the caller owner fails on a Windows host. +const CALLER_OWNER_ON_WINDOWS: &str = "the caller owner requires a Linux host; Windows has no \ + per-request POSIX identity for OpenVMM to switch to"; + +/// The guest targets that a root keeps for its own system or runtime, which no mapping may take. +/// +/// A mapping may be neither at nor below one of the [`trees`](Self::trees), nor at one of their +/// ancestors, where its bind would shadow the tree, nor at one of the [`exact`](Self::exact) +/// paths, though it may be below them. No mapping is ever at the guest's root, `/`. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct ReservedTargets<'a> { + /// Guest trees that the root owns. + pub trees: &'a [&'a str], + /// Guest paths at which the root mounts or creates something, below which mappings may lie. + pub exact: &'a [&'a str], +} + +impl ReservedTargets<'static> { + /// The blockless root, in which the guest's own system runs: [`RESERVED_TARGETS`]. + pub const BLOCKLESS_ROOT: Self = Self { + trees: &RESERVED_TARGETS, + exact: &[], + }; + + /// The container root of a layered sandbox: the trees in which its runtime mounts, binds, or + /// stages something, [`RESERVED_MOUNT_TARGETS`], and exactly the directories on the way to + /// them that it creates, [`RESERVED_EXACT_MOUNT_TARGETS`], which the guest checks as well. + pub const CONTAINER_ROOT: Self = Self { + trees: &RESERVED_MOUNT_TARGETS, + exact: &RESERVED_EXACT_MOUNT_TARGETS, + }; +} + +impl<'a> ReservedTargets<'a> { + /// Returns the reserved path that a mapping at `target`, an absolute guest path, would take + /// or shadow, or `None` if `target` is free. + pub fn reservation(&self, target: &str) -> Option<&'a str> { + if target == "/" { + return Some("/"); + } + self.exact + .iter() + .copied() + .find(|exact| *exact == target) + .or_else(|| { + self.trees + .iter() + .copied() + .find(|tree| within(target, tree) || within(tree, target)) + }) + } +} + +/// The host identity under which OpenVMM runs the file operations of an export. +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Hash, Serialize, Deserialize)] +#[serde(rename_all = "lowercase")] +pub enum Owner { + /// Every operation runs as OpenVMM, the default. + #[default] + Vmm, + /// Each operation runs as the identity of the guest caller, with guest root squashed to the + /// owner of the exported directory or file. Linux hosts only. + Caller, +} + +impl Owner { + /// Returns whether this is the default owner, which the serde form of a plan leaves out. + fn is_vmm(&self) -> bool { + *self == Self::Vmm + } +} + +/// The host paths of one sandbox, as [`plan`] exports them: the children of OpenVMM's aggregate +/// export, the bind mounts of the guest, and the host identity of the file operations. +/// +/// Callers record the plan, so its serde form is part of their state: a caller versions its +/// records, and a record of another version is not read. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct ExportPlan { + /// Children of the aggregate export; each is named by its index. + pub children: Vec, + /// Bind mounts the guest creates, parents before children. + pub binds: Vec, + /// The host identity of the file operations; a plan of the default owner leaves it out. + #[serde(default, skip_serializing_if = "Owner::is_vmm")] + pub owner: Owner, +} + +/// One host directory or file of the aggregate export. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct ExportChild { + /// Canonical host directory, or the canonical host file that the child exposes alone. + pub root: PathBuf, + /// Whether the root is a regular file, which OpenVMM exposes without its directory; a child + /// of a directory leaves it out. + #[serde(default, skip_serializing_if = "std::ops::Not::not")] + pub file: bool, + /// Whether the guest may modify it, which a read-write mapping inside it requires. + pub writable: bool, + /// Canonical host paths inside the child that OpenVMM hides; the root itself, when it shows + /// only its allowed paths. + pub denied: Vec, + /// Identities, at provision, of the denied paths that a workload could move: those inside a + /// writable part. + pub denied_identities: Vec>, + /// Canonical paths inside denied paths that stay visible. + pub allowed: Vec, + /// Canonical paths to which OpenVMM limits the guest's writes; empty if the guest may modify + /// everything that it can reach in a writable child. + pub write: Vec, +} + +/// Device and file index of a host object, which survive renames. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct FileIdentity { + /// Device, or the volume serial number on Windows. + pub device: u64, + /// Inode, or the file index on Windows. + pub index: u64, +} + +/// One mapped host path. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +pub struct Bind { + /// Index of the child that holds the mapped path. + pub child: usize, + /// Path relative to the child's root, `/`-separated; empty for the root itself. + pub source: String, + /// Absolute guest path. + pub target: String, + /// Whether the guest mounts it read-only. + pub read_only: bool, + /// Identity of the host object at provision, when it lies inside a writable part and a + /// workload could therefore move it. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub identity: Option, +} + +impl Bind { + /// Path of the mapped object relative to the guest's mount of the export. + pub fn guest_source(&self) -> String { + guest_source(self.child, &self.source) + } +} + +/// Where the guest sees a mapped host path. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Target<'a> { + /// At the guest path that [`fn@guest_path`] derives from the resolved host path. + Convention, + /// At this absolute guest path: a path other than the root, of at most 4,095 bytes, without + /// NUL characters or empty, `.`, or `..` components. + Guest(&'a str), +} + +/// A host directory or regular file that the guest reads, or reads and writes, at a target. +#[derive(Debug, Clone, Copy)] +pub struct Mapping<'a> { + /// How messages name the list or option that requested the mapping, such as + /// `filesystem.readonlyPaths`. + pub name: &'a str, + /// The host path. + pub path: &'a Path, + /// Where the guest sees it. + pub target: Target<'a>, + /// Whether the guest may only read it. + pub read_only: bool, +} + +impl<'a> Mapping<'a> { + /// Returns the mappings of the paths of `list`, under its name, at the guest paths of the + /// convention, read-only or read-write. + pub fn convention(list: PathList<'a>, read_only: bool) -> impl Iterator + 'a { + list.paths.iter().map(move |path| Self { + name: list.name, + path, + target: Target::Convention, + read_only, + }) + } +} + +/// A list of host paths, and the name by which messages refer to its entries, such as +/// `readOnly`. +#[derive(Debug, Clone, Copy)] +pub struct PathList<'a> { + /// The name of the list in messages. + pub name: &'a str, + /// The host paths. + pub paths: &'a [PathBuf], +} + +impl PathList<'static> { + /// A list without paths. + pub const EMPTY: Self = Self { + name: "", + paths: &[], + }; +} + +/// The host paths that a sandbox maps, those that it hides inside them and shows again inside +/// those, the paths to which it narrows writes, the host identity of the file operations, and the +/// guest targets that the guest's root keeps for itself. +#[derive(Debug, Clone, Copy)] +pub struct ExportRequest<'a> { + /// The mapped host paths, each at its target with its access. + pub mappings: &'a [Mapping<'a>], + /// Host paths hidden inside the mapped paths. A mapped directory may be denied itself when + /// allowed paths inside it stay visible. + pub denied: PathList<'a>, + /// Host paths inside denied paths that stay visible, with the directories on the way to + /// them listing only them. + pub allowed: PathList<'a>, + /// Host paths to which the guest's writes are narrowed inside the read-write directory that + /// they lie in, which must be mapped whole: the guest may then write only them and the + /// read-write mappings inside that directory. + pub write: PathList<'a>, + /// The host identity of the file operations. + pub owner: Owner, + /// The guest targets that the guest's root keeps for itself. + pub reserved: ReservedTargets<'a>, +} + +impl<'a> ExportRequest<'a> { + /// Returns the request that maps `mappings` into a root that reserves `reserved`, without + /// denied, allowed, or write paths, as the VMM. + pub fn new(mappings: &'a [Mapping<'a>], reserved: ReservedTargets<'a>) -> Self { + Self { + mappings, + denied: PathList::EMPTY, + allowed: PathList::EMPTY, + write: PathList::EMPTY, + owner: Owner::Vmm, + reserved, + } + } +} + +/// Plans the export of `request`, or returns `None` if it maps nothing. +/// +/// Mapped paths must exist, and resolve to paths that are valid UTF-8. A mapping's target is the +/// one that the request gives, or the guest path that the convention derives from the resolved +/// host path, so two spellings of one object (links, letter case, or short names) map to one guest +/// path; either must be free in the request's [`ReservedTargets`]. An object mapped twice at one +/// target is mapped once, read-only if either mapping is. Two objects never share a target, and a +/// target inside another mapping's must show the very object that the other mapping shows there. +/// A whole volume cannot be mapped, and a policy that would need more children than OpenVMM +/// offers is refused. +/// +/// A read-only path must not lie inside a writable part, where the host could not keep it +/// read-only: inside a read-write directory, unless write paths narrow that directory's writes +/// to other paths, or inside a write path. A read-write path may lie inside a read-only +/// directory. A write path must lie inside a read-write directory that a child exports whole. +/// +/// A denied path must not hide a mapped path, except that a denied mapped directory shows the +/// allowed paths inside it, unless it is a read-write mapping inside a directory that is not +/// writable as a whole: OpenVMM narrows that directory's writes to the mapping, and it cannot make +/// a hidden path writable. A denied path that does not exist yet must not lie inside a mapped +/// path, because nothing could hide it once a workload creates it; denied paths outside the +/// mapped paths are ignored. An allowed path must lie inside a denied path, with no other allowed +/// path between them, and a write path must not be hidden. The caller owner needs a Linux host, +/// on which no exported directory or file may belong to UID 0 or GID 0. +/// +/// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) when the request breaks +/// these rules or exceeds a limit of OpenVMM or the guest agent. +pub fn plan(request: &ExportRequest<'_>) -> Result> { + let unsupported = |message: String| Err(Error::unsupported(message)); + if request.owner == Owner::Caller && cfg!(windows) { + return unsupported(CALLER_OWNER_ON_WINDOWS.to_owned()); + } + let mut mapped: Vec> = Vec::new(); + for mapping in request.mappings { + let entry = resolve_mapping(mapping, request.reserved)?; + match mapped + .iter_mut() + .find(|other| other.canonical == entry.canonical && other.target == entry.target) + { + // The same object listed twice at one target keeps its most restrictive access, and + // the name and spelling of the entry that gave it. + Some(other) => { + if entry.read_only && !other.read_only { + *other = entry; + } + } + None => mapped.push(entry), + } + } + if mapped.is_empty() { + return Ok(None); + } + if mapped.len() > MAX_HOST_MAPPINGS { + return unsupported(format!( + "the openvmm backend maps at most {MAX_HOST_MAPPINGS} paths into one sandbox" + )); + } + check_targets(&mapped)?; + let roots = outermost(mapped.iter().map(|entry| entry.canonical.clone()).collect()); + let write = write_paths(request.write, &mapped, &roots)?; + check_read_only(&mapped, &write, request.write.name, &roots)?; + distinct_roots(&roots)?; + if roots.len() > MAX_CHILDREN { + // Exporting directories for some of the files would grant more than the files. + return unsupported(format!( + "the mapped paths need {} exported directories and files, but OpenVMM exports at \ + most {MAX_CHILDREN}; map directories that contain the files instead", + roots.len() + )); + } + let allowed = allowed_paths(request.allowed, &roots)?; + let denied = denied_paths(request.denied, &mapped, &allowed)?; + check_allowed_paths(request.allowed.name, request.denied.name, &allowed, &denied)?; + check_write_paths( + request.write.name, + request.denied.name, + &write, + &denied, + &allowed, + )?; + let denied = canonicals(&denied); + let mut allowed = canonicals(&allowed); + allowed.sort(); + + let parts: Vec = roots + .iter() + .map(|root| writable_part(root, &mapped, &write)) + .collect(); + // A read-write mapping inside a child that is not writable as a whole becomes one of the + // child's write paths, and OpenVMM refuses to hide a write path. The write paths of the + // request were checked already. + for (root, part) in roots.iter().zip(&parts) { + for path in &part.write { + if let Some((rule, true)) = nearest_rule(path, &denied, &allowed, true) { + let (name, given) = mapped + .iter() + .find(|entry| entry.canonical == *path) + .map_or((request.write.name, path.as_path()), |entry| { + (entry.name, entry.given.as_path()) + }); + return unsupported(format!( + "{} entry {} hides the {name} entry {}, to which OpenVMM narrows the writes \ + of {}, and OpenVMM cannot make a hidden path writable; map read-write the \ + paths that it shows again instead", + request.denied.name, + rule.display(), + given.display(), + root.display() + )); + } + } + } + // Only objects inside a writable part can be swapped by a workload; pinning others would + // make ordinary host edits, which often replace files, block the next start. + let movable = |path: &Path| { + roots + .iter() + .zip(&parts) + .any(|(root, part)| path.starts_with(root) && part.moves(root, path)) + }; + let mut children = Vec::with_capacity(roots.len()); + let mut policy_bytes = 0; + for (root, part) in roots.iter().zip(&parts) { + // Every root is a mapping itself, so the guest reaches all of it. + let Some(own) = mapped.iter().find(|entry| entry.canonical == *root) else { + return Err(Error::other(format!( + "{} is exported but not mapped", + root.display() + ))); + }; + let inside = |paths: &[PathBuf]| -> Vec { + paths + .iter() + .filter(|path| path.starts_with(root)) + .cloned() + .collect() + }; + let child_allowed = inside(&allowed); + let child_denied = needed_denied_paths(&inside(&denied), &child_allowed); + check_root(root)?; + policy_bytes += check_policy_paths(root, &child_denied, "hide")?; + policy_bytes += check_policy_paths(root, &child_allowed, "show again")?; + policy_bytes += check_policy_paths(root, &part.write, "make writable")?; + let denied_identities = child_denied + .iter() + .map(|path| movable(path).then(|| identity(path)).transpose()) + .collect::>>()?; + children.push(ExportChild { + root: root.clone(), + file: !own.directory, + writable: part.writable, + denied: child_denied, + denied_identities, + allowed: child_allowed, + write: part.write.clone(), + }); + } + if policy_bytes > MAX_AGGREGATE_POLICY_BYTES { + return unsupported(format!( + "the paths that OpenVMM is to hide, show again, or make writable exceed its \ + {MAX_AGGREGATE_POLICY_BYTES}-byte limit" + )); + } + + let mut binds = Vec::with_capacity(mapped.len()); + for entry in &mapped { + let Some(child) = children + .iter() + .position(|child| entry.canonical.starts_with(&child.root)) + else { + return Err(Error::other(format!( + "{} lies in no exported directory", + entry.given.display() + ))); + }; + let Some(source) = relative_text(&children[child].root, &entry.canonical) else { + return unsupported(format!("{} is not valid UTF-8", entry.given.display())); + }; + if GUEST_EXPORT.len() + 1 + guest_source(child, &source).len() > MAX_GUEST_PATH + || entry.target.len() > MAX_GUEST_PATH + { + return unsupported(format!( + "{} is too long for a guest path", + entry.given.display() + )); + } + binds.push(Bind { + child, + source, + target: entry.target.clone(), + read_only: entry.read_only, + identity: movable(&entry.canonical) + .then(|| identity(&entry.canonical)) + .transpose()?, + }); + } + binds.sort_by(|left, right| { + depth(&left.target) + .cmp(&depth(&right.target)) + .then_with(|| left.target.cmp(&right.target)) + }); + if request.owner == Owner::Caller { + check_caller_roots(&children)?; + } + let plan = ExportPlan { + children, + binds, + owner: request.owner, + }; + // Callers read a record of the plan back through `check`, which accepts every plan that this + // function makes. + debug_assert!( + plan.check(request.reserved).is_ok(), + "the planner made a plan that its check refuses: {:?}", + plan.check(request.reserved) + ); + Ok(Some(plan)) +} + +/// Resolves one requested mapping: its host path must exist as a directory or regular file that +/// is not a whole volume, and its target must be a guest path that `reserved` leaves free. +fn resolve_mapping<'a>(mapping: &Mapping<'a>, reserved: ReservedTargets<'_>) -> Result> { + let unsupported = |message: String| Err(Error::unsupported(message)); + let Mapping { + name, + path, + target, + read_only, + } = *mapping; + let canonical = canonicalize(path).map_err(|error| { + Error::unsupported(format!("{name} entry {} is not accessible", path.display())) + .with_source(error) + })?; + // OpenVMM's options and the records of a plan carry host paths as text. + if canonical.to_str().is_none() { + return unsupported(format!( + "{name} entry {} resolves to a path that is not valid UTF-8", + path.display() + )); + } + let metadata = fs::metadata(&canonical).map_err(|error| { + Error::unsupported(format!("cannot inspect {}", path.display())).with_source(error) + })?; + if !metadata.is_dir() && !metadata.is_file() { + return unsupported(format!( + "{name} entry {} is neither a directory nor a regular file", + path.display() + )); + } + if metadata.is_dir() + && let Some(reason) = volume_root_refusal(&canonical) + { + return unsupported(format!("{name} entry {} {reason}", path.display())); + } + let target = match target { + Target::Convention => { + let Some(target) = guest_path(&canonical) else { + return unsupported(format!( + "{name} entry {} has no guest path; use an absolute local path", + path.display() + )); + }; + if let Some(reserved) = reserved.reservation(&target) { + return unsupported(format!( + "{name} entry {} would cover the guest's {reserved}", + path.display() + )); + } + target + } + Target::Guest(target) => { + if let Some(problem) = target_problem(target) { + return unsupported(format!( + "{name} entry {} has the guest target {target:?}, which {problem}", + path.display() + )); + } + if let Some(reserved) = reserved.reservation(target) { + return unsupported(format!( + "{name} entry {} has the guest target {target}, which would cover the \ + guest's {reserved}", + path.display() + )); + } + target.to_owned() + } + }; + Ok(Mapped { + name, + given: path.to_path_buf(), + canonical, + target, + read_only, + directory: metadata.is_dir(), + }) +} + +/// Explains why `target` cannot be a guest target, or returns `None` if it can: it must be an +/// absolute path other than the root, of at most [`MAX_GUEST_PATH`] bytes, without NUL characters +/// or empty, `.`, or `..` components. +fn target_problem(target: &str) -> Option<&'static str> { + let Some(rest) = target.strip_prefix('/') else { + return Some("is not absolute"); + }; + if rest.is_empty() { + return Some("is the guest's root"); + } + if target.len() > MAX_GUEST_PATH { + return Some("is longer than a guest path may be"); + } + if target.contains('\0') { + return Some("contains a NUL character"); + } + if rest + .split('/') + .any(|component| matches!(component, "" | "." | "..")) + { + return Some("has an empty, `.`, or `..` component"); + } + None +} + +/// Refuses mappings whose targets collide: two objects at one target, or a target inside another +/// mapping's that does not show the very object that the other mapping shows there, which it +/// would hide. +fn check_targets(mapped: &[Mapped<'_>]) -> Result<()> { + for (index, entry) in mapped.iter().enumerate() { + for other in &mapped[..index] { + // The entries of one object at one target are merged already. + if entry.target == other.target { + return Err(Error::unsupported(format!( + "{} entry {} and the {} entry {} have the same guest target {}", + entry.name, + entry.given.display(), + other.name, + other.given.display(), + entry.target + ))); + } + let (outer, inner) = if within(&entry.target, &other.target) { + (other, entry) + } else if within(&other.target, &entry.target) { + (entry, other) + } else { + continue; + }; + if !shows(outer, inner) { + return Err(Error::unsupported(format!( + "{} entry {} at {} would hide part of the {} entry {} at {}; map it at a \ + target outside that one", + inner.name, + inner.given.display(), + inner.target, + outer.name, + outer.given.display(), + outer.target + ))); + } + } + } + Ok(()) +} + +/// Returns whether `inner`, whose target lies below that of `outer`, maps the host object that +/// `outer` shows at that target: whether the rest of `inner`'s target spells, name for name, the +/// host path from `outer`'s object to `inner`'s, as a plan's bind sources spell it. A Windows host +/// reads a backslash in a name as a separator, and a name with a root or a drive as another path, +/// so such a guest name never matches. +fn shows(outer: &Mapped<'_>, inner: &Mapped<'_>) -> bool { + let Some(rest) = inner + .target + .strip_prefix(outer.target.as_str()) + .and_then(|rest| rest.strip_prefix('/')) + else { + return false; + }; + outer.directory + && relative_text(&outer.canonical, &inner.canonical).is_some_and(|names| names == rest) +} + +/// Refuses an exported directory or file that belongs to UID 0 or GID 0, to which the caller +/// owner would squash guest root. +fn check_caller_roots(children: &[ExportChild]) -> Result<()> { + #[cfg(unix)] + for child in children { + use std::os::unix::fs::MetadataExt; + + let metadata = fs::metadata(&child.root).map_err(|error| { + Error::unsupported(format!("cannot inspect {}", child.root.display())) + .with_source(error) + })?; + if metadata.uid() == 0 || metadata.gid() == 0 { + return Err(Error::unsupported(format!( + "the caller owner squashes guest root to the owner of each exported directory or \ + file, so {} must not be owned by UID 0 or GID 0", + child.root.display() + ))); + } + } + #[cfg(not(unix))] + let _ = children; + Ok(()) +} + +/// Explains why a mapped host directory, given in canonical form, cannot be exported, or returns +/// `None` if it can: a whole volume. It inspects only the path, not the host. +fn volume_root_refusal(canonical: &Path) -> Option<&'static str> { + (!canonical + .components() + .any(|component| matches!(component, Component::Normal(_)))) + .then_some( + "is a whole volume, which the openvmm backend does not map; map the directories inside \ + it", + ) +} + +/// Explains why `path`, a host path of a recorded plan, is not in the form in which [`plan`] +/// resolves paths, or returns `None` if it is: without a NUL character, which no name holds and +/// no process argument carries; without a `..` component, through which a path that lies inside a +/// directory by its text could leave it; and, on Windows, on a local drive, without the verbatim +/// prefix. It inspects only the path, not the host. +fn resolved_path_problem(path: &Path) -> Option<&'static str> { + if path.as_os_str().as_encoded_bytes().contains(&0) { + return Some("contains a NUL character"); + } + if path + .components() + .any(|component| component == Component::ParentDir) + { + return Some("has a `..` component"); + } + if cfg!(windows) + && !matches!( + path.components().next(), + Some(Component::Prefix(prefix)) if matches!(prefix.kind(), Prefix::Disk(_)) + ) + { + return Some("is not a path on a local drive"); + } + None +} + +/// Sorts `paths`, drops duplicates, and returns those that no other of them contains. +fn outermost(mut paths: Vec) -> Vec { + paths.sort(); + paths.dedup(); + paths + .iter() + .filter(|path| { + !paths + .iter() + .any(|other| other != *path && path.starts_with(other)) + }) + .cloned() + .collect() +} + +/// Canonicalized, validated form of one requested host mapping. +struct Mapped<'a> { + /// How messages name the list or option that requested it. + name: &'a str, + /// Host path exactly as the caller supplied it, for diagnostics. + given: PathBuf, + /// Link-resolved host path used to construct the export. + canonical: PathBuf, + /// Absolute guest destination of the bind mount. + target: String, + /// Whether the guest must mount this path read-only. + read_only: bool, + /// Whether the mapped host object is a directory rather than a regular file. + directory: bool, +} + +#[cfg(test)] +mod tests; diff --git a/nvx-rs/src/host/filesystem/policy.rs b/nvx-rs/src/host/filesystem/policy.rs new file mode 100644 index 000000000..8070124a7 --- /dev/null +++ b/nvx-rs/src/host/filesystem/policy.rs @@ -0,0 +1,491 @@ +//! Resolution of the denied, allowed, and writable paths of an export. + +use std::path::{Path, PathBuf}; + +use super::host_path::{canonicalize, canonicalize_lenient, identity}; +use super::{ExportChild, FileIdentity, Mapped, PathList, outermost}; +use crate::error::{Error, Result}; + +/// A path of an access policy, resolved. +pub(super) struct PolicyPath { + /// Host path as the caller supplied it, for diagnostics. + given: PathBuf, + /// Link-resolved host path. + canonical: PathBuf, +} + +/// Returns the resolved paths of `paths`. +pub(super) fn canonicals(paths: &[PolicyPath]) -> Vec { + paths.iter().map(|path| path.canonical.clone()).collect() +} + +/// Resolves the path of the list `field`, which must exist. +fn resolve_policy_path(field: &str, path: &Path) -> Result { + canonicalize(path).map_err(|error| { + Error::unsupported(format!( + "{field} entry {} is not accessible", + path.display() + )) + .with_source(error) + }) +} + +/// Resolves the write paths of `list`: each must lie inside a read-write directory that a child +/// of `roots` exports whole, other than that directory itself. +pub(super) fn write_paths( + list: PathList<'_>, + mapped: &[Mapped<'_>], + roots: &[PathBuf], +) -> Result> { + let field = list.name; + let mut paths = Vec::with_capacity(list.paths.len()); + for path in list.paths { + let canonical = resolve_policy_path(field, path)?; + let narrows = roots + .iter() + .find(|root| canonical.starts_with(root)) + .is_some_and(|root| { + *root != canonical + && mapped.iter().any(|entry| { + entry.canonical == *root && entry.directory && !entry.read_only + }) + }); + if !narrows { + return Err(Error::unsupported(format!( + "{field} entry {} lies in no directory that is mapped read-write as a whole, \ + whose writes it could narrow", + path.display() + ))); + } + paths.push(PolicyPath { + given: path.clone(), + canonical, + }); + } + Ok(paths) +} + +/// Refuses a read-only mapping that the host could not keep read-only: one inside a write path, +/// one that a read-write mapping holds, unless write paths narrow that mapping's writes, or one of +/// an object that a read-write mapping maps elsewhere. OpenVMM can limit a child's writes to some +/// of its paths, but cannot keep a path inside them read-only. Only the guest's bind mount would +/// then protect the path, and guest root can bypass it: through the export, or on a Windows host, +/// which opens files case-insensitively, through another spelling of a file's name. +pub(super) fn check_read_only( + mapped: &[Mapped<'_>], + write: &[PolicyPath], + write_name: &str, + roots: &[PathBuf], +) -> Result<()> { + // Write paths narrow the writes of the read-write directory that a child exports whole. + let narrowed = |canonical: &Path| { + roots.iter().any(|root| root == canonical) + && write + .iter() + .any(|path| path.canonical.starts_with(canonical)) + }; + let inside = |inner: &Mapped<'_>, outer_name: &str, outer: &Path, instead: &str| { + Err(Error::unsupported(format!( + "{} entry {} lies inside the {outer_name} entry {}, where the host cannot keep it \ + read-only; instead, {instead}", + inner.name, + inner.given.display(), + outer.display() + ))) + }; + for inner in mapped.iter().filter(|entry| entry.read_only) { + if let Some(path) = write + .iter() + .find(|path| inner.canonical.starts_with(&path.canonical)) + { + return inside( + inner, + write_name, + &path.given, + "narrow the writes to paths outside it", + ); + } + let Some(outer) = mapped.iter().find(|outer| { + !outer.read_only + && !narrowed(&outer.canonical) + && (outer.canonical == inner.canonical + || (outer.directory && inner.canonical.starts_with(&outer.canonical))) + }) else { + continue; + }; + if outer.canonical == inner.canonical { + return Err(Error::unsupported(format!( + "{} entry {} names the object of the {} entry {}, which the guest may write, so \ + the host cannot keep it read-only", + inner.name, + inner.given.display(), + outer.name, + outer.given.display() + ))); + } + return inside( + inner, + outer.name, + &outer.given, + "map that directory read-only and the paths in it that the guest writes read-write", + ); + } + Ok(()) +} + +/// Refuses roots that name one directory or file: OpenVMM refuses one object exported twice, +/// which bind mounts and hard links make possible. +pub(super) fn distinct_roots(roots: &[PathBuf]) -> Result<()> { + let mut identities: Vec<(FileIdentity, &Path)> = Vec::with_capacity(roots.len()); + for root in roots { + let current = identity(root)?; + if let Some((_, other)) = identities.iter().find(|(other, _)| *other == current) { + return Err(Error::unsupported(format!( + "{} and {} are the same host directory or file; map it once", + other.display(), + root.display() + ))); + } + identities.push((current, root)); + } + Ok(()) +} + +/// Resolves the allowed paths of `list`, which must exist inside a child of `roots`, below its +/// root. +pub(super) fn allowed_paths(list: PathList<'_>, roots: &[PathBuf]) -> Result> { + let field = list.name; + let mut paths: Vec = Vec::with_capacity(list.paths.len()); + for path in list.paths { + let canonical = resolve_policy_path(field, path)?; + if roots.contains(&canonical) { + return Err(Error::unsupported(format!( + "{field} entry {} is an exported path itself rather than a path inside one", + path.display() + ))); + } + if !roots.iter().any(|root| canonical.starts_with(root)) { + return Err(Error::unsupported(format!( + "{field} entry {} lies in no mapped directory", + path.display() + ))); + } + if !paths.iter().any(|other| other.canonical == canonical) { + paths.push(PolicyPath { + given: path.clone(), + canonical, + }); + } + } + Ok(paths) +} + +/// Resolves the denied paths of `list` that exist, and refuses one that hides a mapped path, +/// which `allowed` paths do not show again, or one that does not exist inside a mapped path. +pub(super) fn denied_paths( + list: PathList<'_>, + mapped: &[Mapped<'_>], + allowed: &[PolicyPath], +) -> Result> { + let field = list.name; + let mut resolved = Vec::with_capacity(list.paths.len()); + for path in list.paths { + let (canonical, exists) = canonicalize_lenient(path).map_err(|error| { + Error::unsupported(format!( + "{field} entry {} cannot be resolved", + path.display() + )) + .with_source(error) + })?; + resolved.push((path, canonical, exists)); + } + let denied_paths: Vec = resolved + .iter() + .filter(|(_, _, exists)| *exists) + .map(|(_, canonical, _)| canonical.clone()) + .collect(); + let allowed_paths = canonicals(allowed); + let mut denied: Vec = Vec::with_capacity(denied_paths.len()); + for (path, canonical, exists) in resolved { + if !exists { + if let Some(entry) = mapped + .iter() + .find(|entry| canonical.starts_with(&entry.canonical)) + { + return Err(Error::unsupported(format!( + "{field} entry {} does not exist, so it cannot be hidden inside the mapped \ + path {}", + path.display(), + entry.given.display() + ))); + } + continue; + } + // A mapped directory that is denied itself shows the allowed paths inside it, and nothing + // else, whichever denied path contains it. + let hidden = mapped.iter().find(|entry| { + entry.canonical.starts_with(&canonical) + && match nearest_rule(&entry.canonical, &denied_paths, &allowed_paths, true) { + Some((rule, true)) => { + !(entry.directory + && rule == entry.canonical.as_path() + && allowed_paths.iter().any(|shown| { + *shown != entry.canonical && shown.starts_with(&entry.canonical) + })) + } + _ => false, + } + }); + if let Some(entry) = hidden { + return Err(Error::unsupported(format!( + "{field} entry {} contains the mapped path {}", + path.display(), + entry.given.display() + ))); + } + if !denied.iter().any(|other| other.canonical == canonical) { + denied.push(PolicyPath { + given: path.clone(), + canonical, + }); + } + } + Ok(denied) +} + +/// Refuses an allowed path that is also denied, or whose nearest enclosing policy path is not a +/// denied one, which OpenVMM refuses. +pub(super) fn check_allowed_paths( + allowed_name: &str, + denied_name: &str, + allowed: &[PolicyPath], + denied: &[PolicyPath], +) -> Result<()> { + let (denied_paths, allowed_paths) = (canonicals(denied), canonicals(allowed)); + for path in allowed { + let refused = |problem: String| { + Err(Error::unsupported(format!( + "{allowed_name} entry {} {problem}", + path.given.display() + ))) + }; + if denied_paths.contains(&path.canonical) { + return refused(format!("is also a {denied_name} entry")); + } + match nearest_rule(&path.canonical, &denied_paths, &allowed_paths, false) { + Some((_, true)) => {} + Some((other, false)) => { + return refused(format!( + "lies inside the {allowed_name} entry {}, which shows it already", + other.display() + )); + } + None => { + return refused(format!( + "lies inside no {denied_name} entry, so it has nothing to show again" + )); + } + } + } + Ok(()) +} + +/// Refuses a write path that a denied path hides, which OpenVMM refuses. +pub(super) fn check_write_paths( + write_name: &str, + denied_name: &str, + write: &[PolicyPath], + denied: &[PolicyPath], + allowed: &[PolicyPath], +) -> Result<()> { + let (denied_paths, allowed_paths) = (canonicals(denied), canonicals(allowed)); + for path in write { + if let Some((rule, true)) = + nearest_rule(&path.canonical, &denied_paths, &allowed_paths, true) + { + return Err(Error::unsupported(format!( + "{write_name} entry {} lies inside the {denied_name} entry {}, which hides it", + path.given.display(), + rule.display() + ))); + } + } + Ok(()) +} + +/// Returns the denied paths of one child that OpenVMM needs to hide, in order: those that no other +/// of them hides already, without an allowed path between them. OpenVMM refuses the others. +/// +/// It takes one child's paths alone: a denied path outside the child, such as one that contains a +/// denied root, hides none of the child for OpenVMM, which receives each child's policy alone. +pub(super) fn needed_denied_paths(denied: &[PathBuf], allowed: &[PathBuf]) -> Vec { + let mut needed: Vec = denied + .iter() + .filter(|path| { + !nearest_rule(path, denied, allowed, false).is_some_and(|(_, denied)| denied) + }) + .cloned() + .collect(); + needed.sort(); + needed +} + +/// Explains how the policy of `child` breaks a rule of OpenVMM, or returns `None` if it keeps them +/// all: it lists no path twice, hides no path inside another that it hides without an allowed +/// path between them, shows each allowed path again inside a hidden path without another allowed +/// path between them, and makes writable only paths strictly inside it that it shows, none inside +/// another. [`crate::host::filesystem::plan`] checks these rules on the paths that it resolves. It reads nothing from the +/// host. +pub(super) fn policy_rule_problem(child: &ExportChild) -> Option { + let (denied, allowed) = (&child.denied, &child.allowed); + for (paths, kind) in [ + (denied, "hidden"), + (allowed, "allowed"), + (&child.write, "writable"), + ] { + let repeated = paths + .iter() + .enumerate() + .find(|&(position, path)| paths[..position].contains(path)); + if let Some((_, path)) = repeated { + return Some(format!("it lists the {kind} path {} twice", path.display())); + } + } + for path in denied { + if let Some((rule, true)) = nearest_rule(path, denied, allowed, false) { + return Some(format!( + "it hides {} inside {}, which it hides already", + path.display(), + rule.display() + )); + } + } + for path in allowed { + if denied.contains(path) { + return Some(format!("it both hides and shows {}", path.display())); + } + match nearest_rule(path, denied, allowed, false) { + Some((_, true)) => {} + Some((rule, false)) => { + return Some(format!( + "it shows {} again inside {}, which it shows already", + path.display(), + rule.display() + )); + } + None => { + return Some(format!( + "it shows {} again, but hides no path that contains it", + path.display() + )); + } + } + } + for path in &child.write { + if *path == child.root { + return Some(format!( + "it makes all of {} writable through a writable path", + path.display() + )); + } + if let Some(other) = child + .write + .iter() + .find(|other| *other != path && path.starts_with(other)) + { + return Some(format!( + "it makes {} writable inside {}, which it makes writable already", + path.display(), + other.display() + )); + } + if let Some((rule, true)) = nearest_rule(path, denied, allowed, true) { + return Some(format!( + "it makes {} writable, but {} hides it", + path.display(), + rule.display() + )); + } + } + None +} + +/// Returns the nearest of the `denied` and `allowed` paths that contains `path`, which counts +/// itself only when `inclusive`, and whether it is a denied path: the rule by which OpenVMM +/// decides whether the guest sees `path`. +pub(super) fn nearest_rule<'p>( + path: &Path, + denied: &'p [PathBuf], + allowed: &'p [PathBuf], + inclusive: bool, +) -> Option<(&'p Path, bool)> { + denied + .iter() + .map(|rule| (rule, true)) + .chain(allowed.iter().map(|rule| (rule, false))) + .filter(|(rule, _)| (inclusive || rule.as_path() != path) && path.starts_with(rule)) + .max_by_key(|(rule, _)| rule.components().count()) + .map(|(rule, denied)| (rule.as_path(), denied)) +} + +/// Whether a child is writable, and the paths to which OpenVMM limits its writes. +pub(super) struct Writable { + /// Whether the guest may modify the child. + pub(super) writable: bool, + /// The paths to which OpenVMM limits the guest's writes; empty if the guest may modify all of + /// a writable child. + pub(super) write: Vec, +} + +impl Writable { + /// Returns whether a workload could move `path`, which lies in the child at `root`: whether it + /// lies strictly inside a writable part, in a directory that the workload may change. + pub(super) fn moves(&self, root: &Path, path: &Path) -> bool { + self.writable + && path != root + && (self.write.is_empty() + || self + .write + .iter() + .any(|write| path != write && path.starts_with(write))) + } +} + +/// Returns the writable part of the child at `root`: its read-write mappings, or, when one of them +/// is the whole child, all of it, unless write paths narrow its writes to them and the read-write +/// mappings inside it. +pub(super) fn writable_part(root: &Path, mapped: &[Mapped<'_>], write: &[PolicyPath]) -> Writable { + let read_write: Vec = mapped + .iter() + .filter(|entry| !entry.read_only && entry.canonical.starts_with(root)) + .map(|entry| entry.canonical.clone()) + .collect(); + if read_write.is_empty() { + return Writable { + writable: false, + write: Vec::new(), + }; + } + let narrowing: Vec = write + .iter() + .filter(|path| path.canonical.starts_with(root)) + .map(|path| path.canonical.clone()) + .collect(); + let write = if !read_write.iter().any(|path| path == root) { + // Unless a read-write mapping covers the whole child, OpenVMM limits writes to them. + outermost(read_write) + } else if narrowing.is_empty() { + Vec::new() + } else { + outermost( + narrowing + .into_iter() + .chain(read_write.into_iter().filter(|path| path != root)) + .collect(), + ) + }; + Writable { + writable: true, + write, + } +} diff --git a/nvx-rs/src/host/filesystem/tests.rs b/nvx-rs/src/host/filesystem/tests.rs new file mode 100644 index 000000000..cfc7a6c6c --- /dev/null +++ b/nvx-rs/src/host/filesystem/tests.rs @@ -0,0 +1,1688 @@ +use super::host_path::display_path; +use super::*; +use crate::ErrorKind; +use crate::host::platform; + +/// The lists of a filesystem policy as callers hold them, under the names that messages give +/// them. +#[derive(Debug, Default)] +struct FilesystemPolicy { + readonly_paths: Vec, + readwrite_paths: Vec, + denied_paths: Vec, +} + +fn plan(policy: &FilesystemPolicy) -> Result> { + let mappings: Vec> = Mapping::convention( + PathList { + name: "filesystem.readonlyPaths", + paths: &policy.readonly_paths, + }, + true, + ) + .chain(Mapping::convention( + PathList { + name: "filesystem.readwritePaths", + paths: &policy.readwrite_paths, + }, + false, + )) + .collect(); + super::plan(&ExportRequest { + denied: PathList { + name: "filesystem.deniedPaths", + paths: &policy.denied_paths, + }, + ..ExportRequest::new(&mappings, ReservedTargets::BLOCKLESS_ROOT) + }) +} + +fn root() -> tempfile::TempDir { + let directory = tempfile::tempdir().unwrap(); + for name in [ + "work/src/secret", + "work/out", + "work/private", + "projects/app", + "projects/private", + "build-output", + "tools/bin", + ] { + fs::create_dir_all(directory.path().join(name)).unwrap(); + } + fs::write(directory.path().join("work/config.json"), b"{}").unwrap(); + fs::write(directory.path().join("work/run.sh"), b"true").unwrap(); + directory +} + +fn canonical(path: &Path) -> PathBuf { + canonicalize(path).unwrap() +} + +fn sources(mapping: &ExportPlan) -> Vec<(String, bool)> { + mapping + .binds + .iter() + .map(|bind| (bind.guest_source(), bind.read_only)) + .collect() +} + +fn option_values(arguments: &[String], option: &str) -> Vec { + arguments + .windows(2) + .filter(|pair| pair[0] == option) + .map(|pair| pair[1].clone()) + .collect() +} + +#[test] +fn guest_paths_follow_the_mxc_convention() { + if cfg!(windows) { + assert_eq!( + guest_path(Path::new(r"C:\Work\src")).as_deref(), + Some("/mnt/c/Work/src") + ); + assert_eq!(guest_path(Path::new(r"D:\")).as_deref(), Some("/mnt/d")); + assert_eq!( + guest_path(Path::new(r"\\?\C:\x\.\y")).as_deref(), + Some("/mnt/c/x/y") + ); + assert_eq!(guest_path(Path::new(r"\\server\share\x")), None); + assert_eq!(guest_path(Path::new(r"\x")), None); + } else { + assert_eq!( + guest_path(Path::new("/home/me/src")).as_deref(), + Some("/home/me/src") + ); + } + assert_eq!(guest_path(Path::new("relative")), None); + assert_eq!(guest_path(Path::new("..")), None); +} + +/// Verifies that a guest path is within itself and its ancestors only, comparing whole +/// components, as the reserved targets here and the sandbox's share checks require. +#[test] +fn guest_paths_are_within_themselves_and_their_ancestors() { + for (path, directory) in [ + ("/run", "/run"), + ("/run/nvx", "/run"), + ("/run/nvx/t", "/run"), + ("/", "/"), + ("/run", "/"), + ("/run/nvx", "/"), + ] { + assert!(within(path, directory), "{path} {directory}"); + } + for (path, directory) in [ + ("/runner", "/run"), + ("/run", "/run/nvx"), + ("/srv/run", "/run"), + ("/", "/run"), + ] { + assert!(!within(path, directory), "{path} {directory}"); + } +} + +/// The layout of microsoft/nvx#281: directories that share no directory but the volume's +/// root, each with its own access. +#[test] +fn unrelated_directories_are_exported_with_their_own_access() { + let directory = root(); + let base = directory.path(); + let policy = FilesystemPolicy { + readonly_paths: vec![base.join("tools")], + readwrite_paths: vec![base.join("projects/app"), base.join("build-output")], + ..FilesystemPolicy::default() + }; + let mapping = plan(&policy).unwrap().unwrap(); + let children: Vec<(PathBuf, bool)> = mapping + .children + .iter() + .map(|child| (child.root.clone(), child.writable)) + .collect(); + assert_eq!( + children, + [ + (canonical(&base.join("build-output")), true), + (canonical(&base.join("projects/app")), true), + (canonical(&base.join("tools")), false), + ] + ); + assert!(mapping.children.iter().all(|child| child.write.is_empty() + && child.allowed.is_empty() + && child.denied.is_empty())); + // Parents mount before children, so the shallower targets come first. + assert_eq!( + sources(&mapping), + [ + ("0".to_owned(), false), + ("2".to_owned(), true), + ("1".to_owned(), false) + ] + ); + + let arguments = mapping.openvmm_arguments(); + assert_eq!(arguments[..2], ["--mount-aggregate", GUEST_EXPORT]); + assert_eq!( + option_values(&arguments, "--mount-child"), + [ + format!( + "0,{},rw", + display_path(&canonical(&base.join("build-output"))) + ), + format!( + "1,{},rw", + display_path(&canonical(&base.join("projects/app"))) + ), + format!("2,{},ro", display_path(&canonical(&base.join("tools")))), + ] + ); + assert_eq!(arguments.len(), 8); + assert_eq!(mapping.kernel_token(), "nvx_maps=3"); + let table = mapping.guest_table(); + assert_eq!(table.len(), 3); + assert_eq!( + table[1].target, + guest_path(&canonical(&base.join("tools"))).unwrap() + ); + assert!(table[1].read_only); +} + +/// The layout of microsoft/nvx#283: directories whose names hold commas, each with a denied +/// subdirectory. +#[test] +fn host_paths_may_contain_commas() { + let directory = root(); + let base = directory.path(); + let alpha = base.join("projects/alpha,beta"); + let reports = base.join("reports/2026, final"); + for path in [&alpha, &reports] { + fs::create_dir_all(path.join("secret")).unwrap(); + } + let policy = FilesystemPolicy { + readonly_paths: vec![reports.clone()], + readwrite_paths: vec![alpha.clone()], + denied_paths: vec![alpha.join("secret"), reports.join("secret")], + }; + let mapping = plan(&policy).unwrap().unwrap(); + let children: Vec<(PathBuf, bool, Vec)> = mapping + .children + .iter() + .map(|child| (child.root.clone(), child.writable, child.denied.clone())) + .collect(); + assert_eq!( + children, + [ + ( + canonical(&alpha), + true, + vec![canonical(&alpha.join("secret"))] + ), + ( + canonical(&reports), + false, + vec![canonical(&reports.join("secret"))] + ), + ] + ); + // OpenVMM reads each child's mode after the last comma, so the host paths keep theirs. + let arguments = mapping.openvmm_arguments(); + assert_eq!( + option_values(&arguments, "--mount-child"), + [ + format!("0,{},rw", display_path(&canonical(&alpha))), + format!("1,{},ro", display_path(&canonical(&reports))), + ] + ); + assert_eq!( + option_values(&arguments, "--mount-deny"), + [ + display_path(&canonical(&alpha.join("secret"))), + display_path(&canonical(&reports.join("secret"))), + ] + ); + // So do the guest paths, which the control channel carries with their lengths. + let mut table: Vec<(String, bool)> = mapping + .guest_table() + .into_iter() + .map(|map| (map.target, map.read_only)) + .collect(); + table.sort(); + assert_eq!( + table, + [ + (guest_path(&canonical(&alpha)).unwrap(), false), + (guest_path(&canonical(&reports)).unwrap(), true), + ] + ); + assert!(table[0].0.ends_with("/projects/alpha,beta"), "{table:?}"); + assert!(table[1].0.ends_with("/reports/2026, final"), "{table:?}"); +} + +#[test] +fn mapped_files_are_exported_alone() { + let directory = root(); + let base = directory.path(); + let policy = FilesystemPolicy { + readonly_paths: vec![base.join("work/config.json"), base.join("work/src")], + readwrite_paths: vec![base.join("work/run.sh"), base.join("work/out")], + denied_paths: vec![ + // Not exported at all: no child holds it. + base.join("work/private"), + base.join("work/src/secret"), + ], + }; + let mapping = plan(&policy).unwrap().unwrap(); + let work = canonical(&base.join("work")); + // Each file is a child of its own, without its directory, beside the directories. + let children: Vec<(PathBuf, bool, bool)> = mapping + .children + .iter() + .map(|child| (child.root.clone(), child.file, child.writable)) + .collect(); + assert_eq!( + children, + [ + (work.join("config.json"), true, false), + (work.join("out"), false, true), + (work.join("run.sh"), true, true), + (work.join("src"), false, false), + ] + ); + assert!( + mapping + .children + .iter() + .all(|child| child.allowed.is_empty() && child.write.is_empty()) + ); + assert_eq!( + mapping.children[3].denied, + [work.join("src").join("secret")] + ); + assert_eq!( + sources(&mapping), + [ + ("0".to_owned(), true), + ("1".to_owned(), false), + ("2".to_owned(), false), + ("3".to_owned(), true), + ] + ); + + // OpenVMM exposes each file as the child itself, so nothing hides its directory, and the + // `file` flag makes it refuse a directory in the file's place. + let arguments = mapping.openvmm_arguments(); + assert_eq!( + option_values(&arguments, "--mount-child")[0], + format!("0,{},ro,file", display_path(&work.join("config.json"))) + ); + assert_eq!( + option_values(&arguments, "--mount-deny"), + [display_path(&work.join("src").join("secret"))] + ); + assert!(option_values(&arguments, "--mount-allow").is_empty()); + assert!(option_values(&arguments, "--mount-write").is_empty()); + + // A record without file children leaves the field out, and one with them names it. + let record = serde_json::to_value(&mapping).unwrap(); + assert_eq!(record["children"][0]["file"], true); + assert!(record["children"][1].get("file").is_none()); + let restored: ExportPlan = serde_json::from_value(record).unwrap(); + assert_eq!(restored, mapping); + + // A file inside a mapped directory belongs to that directory's child, whose writes + // OpenVMM limits to the file when the directory is read-only. + let nested = FilesystemPolicy { + readonly_paths: vec![base.join("work")], + readwrite_paths: vec![base.join("work/config.json")], + ..FilesystemPolicy::default() + }; + let mapping = plan(&nested).unwrap().unwrap(); + assert_eq!(mapping.children.len(), 1); + assert!(!mapping.children[0].file && mapping.children[0].writable); + assert_eq!(mapping.children[0].write, [work.join("config.json")]); + assert_eq!( + sources(&mapping), + [("0".to_owned(), true), ("0/config.json".to_owned(), false)] + ); + + // One file reached through two names is one object, which is mapped once. + let link = base.join("work/config-link.json"); + fs::hard_link(base.join("work/config.json"), &link).unwrap(); + let aliased = FilesystemPolicy { + readonly_paths: vec![base.join("work/config.json"), link], + ..FilesystemPolicy::default() + }; + let error = plan(&aliased).unwrap_err(); + assert!( + error.message().contains("same host directory or file"), + "{error}" + ); +} + +#[test] +fn policies_beyond_the_children_of_an_aggregate_are_refused() { + let directory = root(); + let base = directory.path(); + let files = |parent: &str, count: usize| -> Vec { + (0..count) + .map(|index| { + let path = base.join(parent).join(format!("file-{index:03}")); + fs::write(&path, b"x").unwrap(); + path + }) + .collect() + }; + // Up to 256 files are children of their own, however many share a directory. + let policy = FilesystemPolicy { + readonly_paths: files("tools", 256), + ..FilesystemPolicy::default() + }; + let mapping = plan(&policy).unwrap().unwrap(); + assert_eq!(mapping.children.len(), 256); + assert!( + mapping + .children + .iter() + .all(|child| child.file && child.allowed.is_empty()) + ); + + // More are refused rather than exported through their directory, which would grant + // more than the files. + let mut readonly_paths = files("tools", 256); + readonly_paths.extend(files("build-output", 1)); + let error = plan(&FilesystemPolicy { + readonly_paths, + ..FilesystemPolicy::default() + }) + .unwrap_err(); + assert_eq!(error.kind(), ErrorKind::Unsupported); + assert!(error.message().contains("at most 256"), "{error}"); + assert!( + error + .message() + .contains("map directories that contain the files instead"), + "{error}" + ); + + // A policy that grants the directory instead needs one child. + let mut readonly_paths = vec![base.join("tools")]; + readonly_paths.extend(files("build-output", 1)); + let mapping = plan(&FilesystemPolicy { + readonly_paths, + ..FilesystemPolicy::default() + }) + .unwrap() + .unwrap(); + assert_eq!(mapping.children.len(), 2); +} + +#[test] +fn nested_mappings_merge_into_the_outermost_directory() { + let directory = root(); + let base = directory.path(); + fs::create_dir_all(base.join("work/out/cache")).unwrap(); + let policy = FilesystemPolicy { + readonly_paths: vec![base.join("work"), base.join("work/src")], + readwrite_paths: vec![base.join("work/out"), base.join("work/out/cache")], + ..FilesystemPolicy::default() + }; + let mapping = plan(&policy).unwrap().unwrap(); + assert_eq!(mapping.children.len(), 1); + let child = &mapping.children[0]; + let work = canonical(&base.join("work")); + assert!(child.writable); + // The read-only root is read-only on the host, except the outermost read-write mapping + // inside it, to which OpenVMM limits writes. + assert_eq!(child.write, [work.join("out")]); + assert_eq!( + sources(&mapping), + [ + ("0".to_owned(), true), + ("0/out".to_owned(), false), + ("0/src".to_owned(), true), + ("0/out/cache".to_owned(), false), + ] + ); + + // A read-write root covers everything, so OpenVMM need not limit writes. + let policy = FilesystemPolicy { + readwrite_paths: vec![base.join("work"), base.join("work/src")], + ..FilesystemPolicy::default() + }; + let mapping = plan(&policy).unwrap().unwrap(); + assert!(mapping.children[0].writable && mapping.children[0].write.is_empty()); + assert_eq!( + sources(&mapping), + [("0".to_owned(), false), ("0/src".to_owned(), false)] + ); + + // The read-write duplicate of a read-only path is tightened to read-only, so a + // read-write path inside it is the only writable one. + let policy = FilesystemPolicy { + readonly_paths: vec![base.join("work")], + readwrite_paths: vec![base.join("work"), base.join("work/src")], + ..FilesystemPolicy::default() + }; + let mapping = plan(&policy).unwrap().unwrap(); + assert!(mapping.children[0].writable); + assert_eq!(mapping.children[0].write, [work.join("src")]); + assert_eq!( + sources(&mapping), + [("0".to_owned(), true), ("0/src".to_owned(), false)] + ); +} + +/// Only the guest's bind mount could keep a read-only path inside a read-write directory +/// read-only, so such a path is refused on every host, under any of its names, while the +/// inverse nesting and denied paths inside a read-write directory stay valid. +#[test] +fn read_only_paths_inside_read_write_directories_are_refused() { + let directory = root(); + let base = directory.path(); + fs::create_dir(base.join("work/output")).unwrap(); + let policy = |readonly_paths: &[PathBuf], readwrite_paths: &[PathBuf]| FilesystemPolicy { + readonly_paths: readonly_paths.to_vec(), + readwrite_paths: readwrite_paths.to_vec(), + ..FilesystemPolicy::default() + }; + let refusal = |inner: &Path, outer: &Path| { + format!( + "filesystem.readonlyPaths entry {} lies inside the filesystem.readwritePaths \ + entry {},", + inner.display(), + outer.display() + ) + }; + let work = base.join("work"); + let (src, config) = (work.join("src"), work.join("config.json")); + let secret = src.join("secret"); + // A directory, a file, a path deeper inside, a path that is also listed read-write, and a + // path inside a read-write directory that itself lies inside a read-only one. + for (readonly, readwrite, inner, outer) in [ + (vec![src.clone()], vec![work.clone()], &src, &work), + (vec![config.clone()], vec![work.clone()], &config, &work), + (vec![secret.clone()], vec![work.clone()], &secret, &work), + ( + vec![src.clone()], + vec![work.clone(), src.clone()], + &src, + &work, + ), + ( + vec![work.clone(), secret.clone()], + vec![src.clone()], + &secret, + &src, + ), + ] { + let error = plan(&policy(&readonly, &readwrite)).unwrap_err(); + assert_eq!(error.kind(), ErrorKind::Unsupported, "{error}"); + assert!( + error.message().starts_with(&refusal(inner, outer)), + "{error}" + ); + } + // Resolved paths decide, so another name of a nested path is refused too: a spelling in + // other letter case on Windows, and one through a symbolic link elsewhere. + #[cfg(windows)] + let alias = base.join("WORK").join("CONFIG.JSON"); + #[cfg(unix)] + let alias = { + std::os::unix::fs::symlink(&work, base.join("work-link")).unwrap(); + base.join("work-link").join("config.json") + }; + let error = plan(&policy( + std::slice::from_ref(&alias), + std::slice::from_ref(&work), + )) + .unwrap_err(); + assert!( + error.message().starts_with(&refusal(&alias, &work)), + "{error}" + ); + + // The inverse nesting is valid: OpenVMM limits the read-only directory's writes to the + // read-write path inside it. + let mapping = plan(&policy(std::slice::from_ref(&work), &[work.join("out")])) + .unwrap() + .unwrap(); + assert_eq!(mapping.children[0].write, [canonical(&work.join("out"))]); + // Containment compares whole components, so work/output lies beside work/out. + let mapping = plan(&policy(&[work.join("output")], &[work.join("out")])) + .unwrap() + .unwrap(); + assert_eq!(mapping.children.len(), 2); + let denied = FilesystemPolicy { + readwrite_paths: vec![work.clone()], + denied_paths: vec![work.join("private")], + ..FilesystemPolicy::default() + }; + assert_eq!(plan(&denied).unwrap().unwrap().children[0].denied.len(), 1); +} + +#[test] +fn denied_paths_belong_to_the_directory_that_contains_them() { + let directory = root(); + let base = directory.path(); + let policy = FilesystemPolicy { + readonly_paths: vec![base.join("work/src"), base.join("projects")], + readwrite_paths: vec![base.join("build-output")], + denied_paths: vec![ + base.join("work/src/secret"), + base.join("projects/private"), + base.join("work/out"), + base.join("elsewhere"), + ], + }; + let mapping = plan(&policy).unwrap().unwrap(); + let denied: Vec> = mapping + .children + .iter() + .map(|child| child.denied.clone()) + .collect(); + assert_eq!( + denied, + [ + Vec::new(), + vec![canonical(&base.join("projects/private"))], + vec![canonical(&base.join("work/src/secret"))], + ] + ); + // Nothing a workload could move is pinned without a read-write parent. + assert!( + mapping + .children + .iter() + .flat_map(|child| &child.denied_identities) + .all(Option::is_none) + ); + assert_eq!( + option_values(&mapping.openvmm_arguments(), "--mount-deny").len(), + 2 + ); +} + +/// Checks the order, duplicates, and containment by whole components that the roots, denied +/// paths, and writable paths share, which needs no host paths. +#[test] +fn outermost_paths_are_sorted_once_without_the_paths_inside_others() { + let paths = ["/b/c", "/a/bc", "/b", "/a/b", "/b/c/d", "/a/b"].map(PathBuf::from); + assert_eq!( + outermost(paths.to_vec()), + ["/a/b", "/a/bc", "/b"].map(PathBuf::from) + ); +} + +#[test] +fn unenforceable_policies_are_rejected() { + let directory = root(); + let base = directory.path(); + let cases = [ + FilesystemPolicy { + readonly_paths: vec![base.join("missing")], + ..FilesystemPolicy::default() + }, + FilesystemPolicy { + readwrite_paths: vec![base.join("work/out")], + denied_paths: vec![base.join("work")], + ..FilesystemPolicy::default() + }, + FilesystemPolicy { + readwrite_paths: vec![base.join("work/out")], + denied_paths: vec![base.join("work/out/not-yet")], + ..FilesystemPolicy::default() + }, + FilesystemPolicy { + readonly_paths: vec![base.join("work/config.json")], + denied_paths: vec![base.join("work")], + ..FilesystemPolicy::default() + }, + // A whole volume is never exported. + FilesystemPolicy { + readonly_paths: vec![PathBuf::from(if cfg!(windows) { r"C:\" } else { "/" })], + ..FilesystemPolicy::default() + }, + ]; + for policy in cases { + assert_eq!( + plan(&policy).unwrap_err().kind(), + ErrorKind::Unsupported, + "{policy:?}" + ); + } + if !cfg!(windows) { + let system = FilesystemPolicy { + readonly_paths: vec![PathBuf::from("/usr")], + ..FilesystemPolicy::default() + }; + assert_eq!(plan(&system).unwrap_err().kind(), ErrorKind::Unsupported); + } + let empty = FilesystemPolicy { + denied_paths: vec![base.join("tools")], + ..FilesystemPolicy::default() + }; + assert_eq!(plan(&empty).unwrap(), None); +} + +#[test] +fn whole_volumes_are_refused() { + // The paths are only inspected, so nothing is created at a volume root. A file directly + // in a volume's root is exported alone, so only a whole volume is refused. + let (volumes, directories): (&[&str], &[&str]) = if cfg!(windows) { + (&[r"C:\", r"D:\"], &[r"C:\work", r"D:\a\b"]) + } else { + (&["/"], &["/work", "/srv/a/b"]) + }; + for volume in volumes { + let reason = volume_root_refusal(Path::new(volume)).unwrap(); + assert!(reason.contains("whole volume"), "{volume}: {reason}"); + } + for directory in directories { + assert_eq!(volume_root_refusal(Path::new(directory)), None); + } +} + +#[test] +fn spellings_of_one_object_share_its_guest_path() { + let directory = root(); + let base = directory.path(); + let upper = if cfg!(windows) { + base.join("WORK").join("SRC") + } else { + base.join("work").join(".").join("src") + }; + let policy = FilesystemPolicy { + readonly_paths: vec![upper], + readwrite_paths: vec![base.join("work").join("src")], + ..FilesystemPolicy::default() + }; + let mapping = plan(&policy).unwrap().unwrap(); + assert_eq!(mapping.binds.len(), 1); + assert!(mapping.binds[0].read_only); + assert!(!mapping.children[0].writable); + assert_eq!( + mapping.binds[0].target, + guest_path(&canonical(&base.join("work").join("src"))).unwrap() + ); +} + +#[cfg(windows)] +#[test] +fn resolved_guest_paths_canonicalize_temp_directory_case_aliases() { + let directory = root(); + let path = directory.path().join("work").join("out"); + let upper = PathBuf::from(path.to_str().unwrap().to_uppercase()); + let expected = guest_path(&canonical(&path)).unwrap(); + assert_ne!(guest_path(&upper), Some(expected.clone())); + assert_eq!(resolve_guest_path(&upper), Some(expected)); +} + +#[cfg(windows)] +#[test] +fn paths_that_only_a_verbatim_path_names_are_refused() { + let directory = root(); + let base = directory.path(); + // A verbatim path keeps a trailing period or space of a name, which Windows drops from a + // path without the prefix: there, `work.` and `work ` would name `work`, and `solo.` + // nothing. + let verbatim = fs::canonicalize(base).unwrap(); + let named = |name: &str| PathBuf::from(format!(r"{}\{name}", verbatim.display())); + for name in ["work.", "work ", "solo.", r"work\src\secret."] { + fs::create_dir(named(name)).unwrap(); + } + for name in ["work.", "work ", "solo."] { + let path = named(name); + let error = canonicalize(&path).unwrap_err(); + assert_eq!(error.kind(), std::io::ErrorKind::Unsupported, "{error}"); + let policy = FilesystemPolicy { + readwrite_paths: vec![path], + ..FilesystemPolicy::default() + }; + let error = plan(&policy).unwrap_err(); + assert_eq!(error.kind(), ErrorKind::Unsupported); + assert!(error.to_string().contains("is not accessible"), "{error}"); + } + // A denied path that only a verbatim path names would hide another path instead. + let policy = FilesystemPolicy { + readwrite_paths: vec![base.join("work")], + denied_paths: vec![named(r"work\src\secret.")], + ..FilesystemPolicy::default() + }; + let error = plan(&policy).unwrap_err(); + assert_eq!(error.kind(), ErrorKind::Unsupported); + assert!(error.to_string().contains("cannot be resolved"), "{error}"); + // A verbatim path that names what the path without its prefix names is planned as that + // path. + let policy = FilesystemPolicy { + readwrite_paths: vec![named("work")], + ..FilesystemPolicy::default() + }; + let exported = plan(&policy).unwrap().unwrap(); + assert_eq!(exported.children[0].root, canonical(&base.join("work"))); +} + +#[test] +fn policy_paths_follow_openvmm_rules() { + let directory = root(); + let base = directory.path(); + for name in ["work/out/My Secrets", "work/out/ leading", "my files"] { + fs::create_dir_all(base.join(name)).unwrap(); + } + fs::write(base.join("my files/notes one.txt"), b"notes").unwrap(); + // Names may contain spaces, in a file that a child exposes alone and in a policy path. + let spaced = FilesystemPolicy { + readonly_paths: vec![base.join("my files/notes one.txt")], + readwrite_paths: vec![base.join("work/out")], + denied_paths: vec![base.join("work/out/My Secrets")], + }; + let mapping = plan(&spaced).unwrap().unwrap(); + let arguments = mapping.openvmm_arguments(); + assert!( + option_values(&arguments, "--mount-child")[0].ends_with("notes one.txt,ro,file"), + "{arguments:?}" + ); + assert!(option_values(&arguments, "--mount-deny")[0].ends_with("My Secrets")); + // But they must not begin or end with one. + let leading = FilesystemPolicy { + readwrite_paths: vec![base.join("work/out")], + denied_paths: vec![base.join("work/out/ leading")], + ..FilesystemPolicy::default() + }; + assert_eq!(plan(&leading).unwrap_err().kind(), ErrorKind::Unsupported); + if !cfg!(windows) { + fs::create_dir_all(base.join("work/out/tab\there")).unwrap(); + fs::create_dir_all(base.join("work/out/colon:here")).unwrap(); + for name in ["tab\there", "colon:here"] { + let policy = FilesystemPolicy { + readwrite_paths: vec![base.join("work/out")], + denied_paths: vec![base.join("work/out").join(name)], + ..FilesystemPolicy::default() + }; + assert_eq!( + plan(&policy).unwrap_err().kind(), + ErrorKind::Unsupported, + "{name}" + ); + } + } +} + +#[test] +fn openvmm_limits_are_checked_per_directory() { + let directory = root(); + let base = directory.path(); + let files = |count: usize| -> Vec { + (0..count) + .map(|index| { + let path = base.join("tools").join(format!("tool-{index:03}")); + fs::write(&path, b"x").unwrap(); + path + }) + .collect() + }; + // One exported directory hides at most 128 denied paths, and singles out at most 128 + // read-write paths inside a read-only one. + for policy in [ + FilesystemPolicy { + readonly_paths: vec![base.join("tools")], + denied_paths: files(129), + ..FilesystemPolicy::default() + }, + FilesystemPolicy { + readonly_paths: vec![base.join("tools")], + readwrite_paths: files(129), + ..FilesystemPolicy::default() + }, + ] { + let error = plan(&policy).unwrap_err(); + assert_eq!(error.kind(), ErrorKind::Unsupported); + assert!(error.message().contains("at most 128"), "{error}"); + } +} + +#[test] +fn swapped_objects_fail_verification() { + let directory = root(); + let base = directory.path(); + // Everything below the read-write work directory is pinned. + let policy = FilesystemPolicy { + readwrite_paths: vec![base.join("work"), base.join("work/src")], + denied_paths: vec![base.join("work/src/secret")], + ..FilesystemPolicy::default() + }; + let mapping = plan(&policy).unwrap().unwrap(); + assert!( + mapping.children[0] + .denied_identities + .iter() + .all(Option::is_some) + ); + mapping.verify().unwrap(); + fs::rename(base.join("work/src/secret"), base.join("work/src/moved")).unwrap(); + fs::create_dir(base.join("work/src/secret")).unwrap(); + assert_eq!(mapping.verify().unwrap_err().kind(), ErrorKind::Other); + fs::remove_dir(base.join("work/src/secret")).unwrap(); + fs::rename(base.join("work/src/moved"), base.join("work/src/secret")).unwrap(); + mapping.verify().unwrap(); + fs::rename(base.join("work/src"), base.join("work/src-moved")).unwrap(); + fs::create_dir_all(base.join("work/src/secret")).unwrap(); + assert_eq!(mapping.verify().unwrap_err().kind(), ErrorKind::Other); + + // Without a read-write parent, nothing a workload could move is pinned, so host edits + // that replace objects do not block the next start. + let read_only = FilesystemPolicy { + readonly_paths: vec![base.join("tools")], + ..FilesystemPolicy::default() + }; + let mapping = plan(&read_only).unwrap().unwrap(); + fs::remove_dir_all(base.join("tools")).unwrap(); + fs::create_dir(base.join("tools")).unwrap(); + mapping.verify().unwrap(); + + // Every child keeps its kind: a directory in place of a file child would grant all of it, + // and a directory child must remain a directory. + let settings = base.join("settings.json"); + fs::write(&settings, b"{}").unwrap(); + let with_file = FilesystemPolicy { + readonly_paths: vec![settings.clone(), base.join("tools")], + ..FilesystemPolicy::default() + }; + let mapping = plan(&with_file).unwrap().unwrap(); + mapping.verify().unwrap(); + fs::remove_file(&settings).unwrap(); + fs::create_dir(&settings).unwrap(); + assert_eq!(mapping.verify().unwrap_err().kind(), ErrorKind::Other); + fs::remove_dir(&settings).unwrap(); + fs::write(&settings, b"{}").unwrap(); + mapping.verify().unwrap(); + fs::remove_dir(base.join("tools")).unwrap(); + fs::write(base.join("tools"), b"").unwrap(); + assert_eq!(mapping.verify().unwrap_err().kind(), ErrorKind::Other); +} + +/// A recorded plan with a read-only bind in a part of a child that the guest may write, which +/// only the guest's bind mount would keep read-only, is refused before a guest runs. +#[test] +fn read_only_binds_that_the_host_cannot_enforce_fail_verification() { + let directory = root(); + let base = directory.path(); + fs::create_dir_all(base.join("work/out/cache")).unwrap(); + let refused = |mut mapping: ExportPlan, source: &str| { + let bind = mapping + .binds + .iter_mut() + .find(|bind| bind.source == source) + .unwrap(); + bind.read_only = true; + let error = mapping.verify().unwrap_err(); + assert_eq!(error.kind(), ErrorKind::Unsupported, "{error}"); + assert!( + error.message().contains("read-only inside a part of"), + "{error}" + ); + }; + // A read-only path inside a read-write root, all of which the guest may write. + let writable_root = plan(&FilesystemPolicy { + readwrite_paths: vec![base.join("work"), base.join("work/src")], + ..FilesystemPolicy::default() + }) + .unwrap() + .unwrap(); + writable_root.verify().unwrap(); + refused(writable_root, "src"); + // A read-only path inside a writable path of a read-only root, whose other read-only + // paths stay read-only on the host. + let writable_path = plan(&FilesystemPolicy { + readonly_paths: vec![base.join("work"), base.join("work/src")], + readwrite_paths: vec![base.join("work/out"), base.join("work/out/cache")], + ..FilesystemPolicy::default() + }) + .unwrap() + .unwrap(); + writable_path.verify().unwrap(); + refused(writable_path, "out/cache"); +} + +/// A directory on another volume than the temporary directory, if the host has one: the one +/// that `NVX_TEST_SECOND_VOLUME` names, or the runner's temporary directory +/// on GitHub's Windows runners. Tests create their files below it. Linux's `/dev/shm` does +/// not qualify, because it lies in the guest's reserved `/dev`. +fn second_volume() -> Option { + let candidate = std::env::var_os("NVX_TEST_SECOND_VOLUME") + .or_else(|| std::env::var_os("RUNNER_TEMP")) + .map(PathBuf::from)?; + let temporary = canonical(&std::env::temp_dir()); + let candidate = canonicalize(&candidate).ok()?; + let volume = |path: &Path| platform::file_identity(path).ok().map(|(device, _)| device); + (candidate.is_dir() && volume(&candidate) != volume(&temporary)).then_some(candidate) +} + +#[test] +fn directories_on_other_volumes_are_exported_too() { + let Some(volume) = second_volume() else { + eprintln!("skipped: the host has no second volume"); + return; + }; + let directory = root(); + let other = tempfile::tempdir_in(volume).unwrap(); + fs::create_dir(other.path().join("cache")).unwrap(); + let policy = FilesystemPolicy { + readonly_paths: vec![directory.path().join("tools")], + readwrite_paths: vec![other.path().join("cache")], + ..FilesystemPolicy::default() + }; + let mapping = plan(&policy).unwrap().unwrap(); + assert_eq!(mapping.children.len(), 2); + let modes: Vec = mapping + .children + .iter() + .map(|child| child.writable) + .collect(); + assert!(modes.contains(&true) && modes.contains(&false)); +} +/// Returns a mapping of `path` at the guest path `target`. +fn at<'a>(path: &'a Path, target: &'a str, read_only: bool) -> Mapping<'a> { + Mapping { + name: "mappings", + path, + target: Target::Guest(target), + read_only, + } +} + +#[test] +fn explicit_targets_place_mappings_at_any_free_guest_path() { + let directory = root(); + let base = directory.path(); + let (work, tools) = (base.join("work"), base.join("tools")); + let config = work.join("config.json"); + let mappings = [ + at(&work, "/workspace", false), + at(&tools, "/opt/tools", true), + // A target inside another shows what the other shows there. + at(&config, "/workspace/config.json", false), + // One object may appear at several targets. + at(&work, "/mirror", false), + ]; + let request = ExportRequest::new(&mappings, ReservedTargets::CONTAINER_ROOT); + let mapping = super::plan(&request).unwrap().unwrap(); + let roots: Vec<&Path> = mapping + .children + .iter() + .map(|child| child.root.as_path()) + .collect(); + assert_eq!(roots, [canonical(&tools), canonical(&work)]); + let binds: Vec<(&str, String, bool)> = mapping + .binds + .iter() + .map(|bind| (bind.target.as_str(), bind.guest_source(), bind.read_only)) + .collect(); + assert_eq!( + binds, + [ + ("/mirror", "1".to_owned(), false), + ("/workspace", "1".to_owned(), false), + ("/opt/tools", "0".to_owned(), true), + ("/workspace/config.json", "1/config.json".to_owned(), false), + ] + ); + mapping.check(ReservedTargets::CONTAINER_ROOT).unwrap(); + // A root that keeps its whole tree refuses every mapping, planned or recorded. + let everything = ReservedTargets { + trees: &["/"], + exact: &[], + }; + for error in [ + super::plan(&ExportRequest::new(&mappings, everything)).unwrap_err(), + mapping.check(everything).unwrap_err(), + ] { + assert_eq!(error.kind(), ErrorKind::Unsupported, "{error}"); + assert!( + error.message().ends_with("would cover the guest's /"), + "{error}" + ); + } + // The convention's plans pass the check of their root too. + let convention = plan(&FilesystemPolicy { + readonly_paths: vec![tools.clone()], + readwrite_paths: vec![work.clone(), config.clone()], + ..FilesystemPolicy::default() + }) + .unwrap() + .unwrap(); + convention.check(ReservedTargets::BLOCKLESS_ROOT).unwrap(); + + let refused = |mappings: &[Mapping<'_>], expected: &str| { + let request = ExportRequest::new(mappings, ReservedTargets::CONTAINER_ROOT); + let error = super::plan(&request).unwrap_err(); + assert_eq!(error.kind(), ErrorKind::Unsupported, "{error}"); + assert!( + error.message().contains(expected), + "{error} lacks {expected}" + ); + }; + refused( + &[at(&work, "/data", true), at(&tools, "/data", true)], + "have the same guest target /data", + ); + refused( + &[at(&work, "/data", true), at(&tools, "/data/tools", true)], + "at /data/tools would hide part of the mappings entry", + ); + // A file shows nothing below it. + refused( + &[at(&config, "/data", true), at(&tools, "/data/tools", true)], + "would hide part of", + ); + let long = format!("/{}", "x".repeat(4095)); + for (target, problem) in [ + ("data", "is not absolute"), + ("/", "is the guest's root"), + ("/a//b", "has an empty, `.`, or `..` component"), + ("/a/./b", "has an empty, `.`, or `..` component"), + ("/a/../b", "has an empty, `.`, or `..` component"), + ("/a/", "has an empty, `.`, or `..` component"), + ("/a\0b", "contains a NUL character"), + (long.as_str(), "is longer than a guest path may be"), + ] { + refused(&[at(&work, target, true)], problem); + } + refused( + &[at(&work, "/etc", true)], + "has the guest target /etc, which would cover the guest's /etc", + ); + refused( + &[at(&work, "/proc/self", true)], + "would cover the guest's /proc", + ); + refused( + &[at(&config, "/etc/resolv.conf", true)], + "has the guest target /etc/resolv.conf, which would cover the guest's /etc/resolv.conf", + ); + // The same object read-only and read-write at two targets would let the guest write it. + refused( + &[at(&work, "/a", false), at(&work, "/b", true)], + "names the object of the mappings entry", + ); + // A target inside another must spell, name for name, the host path below the other's + // object, so a backslash in a guest name, which a Windows host reads as a separator, and a + // guest name that a Windows host reads as a root of its own show nothing there. + let host = work.join("src").join("host"); + fs::create_dir_all(&host).unwrap(); + refused( + &[ + at(&work, "/workspace", true), + at(&host, r"/workspace/src\host", true), + ], + "would hide part of", + ); + let resolved = canonical(&work); + let text = resolved.to_str().unwrap(); + let rooted = text + .split_once(':') + .map_or(text, |(_, rest)| rest) + .replace('/', r"\"); + let inside = format!("/opt/tools/{rooted}"); + refused( + &[at(&tools, "/opt/tools", true), at(&work, &inside, true)], + "would hide part of", + ); +} + +#[test] +fn reserved_targets_depend_on_the_root() { + let container = ReservedTargets::CONTAINER_ROOT; + for (target, reserved) in [ + ("/", "/"), + ("/proc", "/proc"), + ("/proc/self", "/proc"), + ("/sys/fs", "/sys"), + ("/dev/shm", "/dev"), + ("/.nvx-agent/tools", "/.nvx-agent"), + ("/etc", "/etc"), + ("/etc/machine-id", "/etc/machine-id"), + ("/etc/resolv.conf", "/etc/resolv.conf"), + ("/run", "/run"), + // The ancestors of the runtime's trees are reserved, as exact paths here. + ("/run/nvx", "/run/nvx"), + ("/run/nvx/time/state", "/run/nvx/time"), + ] { + assert_eq!(container.reservation(target), Some(reserved), "{target}"); + } + for target in [ + "/etc/app", + "/etc/resolvconf", + "/run/secrets", + "/run/nvx/other", + "/usr/local", + "/workspace", + "/procfs", + ] { + assert_eq!(container.reservation(target), None, "{target}"); + } + let blockless = ReservedTargets::BLOCKLESS_ROOT; + for (target, reserved) in [ + ("/", "/"), + ("/usr/local", "/usr"), + ("/run", "/run"), + ("/etc/app", "/etc"), + ("/init", "/init"), + ] { + assert_eq!(blockless.reservation(target), Some(reserved), "{target}"); + } + for target in ["/workspace", "/mnt/c/work", "/home/me", "/initrd", "/opt"] { + assert_eq!(blockless.reservation(target), None, "{target}"); + } + // A mapping at an ancestor of a reserved tree would shadow it. + let nested = ReservedTargets { + trees: &["/srv/runtime/state"], + exact: &[], + }; + assert_eq!(nested.reservation("/srv"), Some("/srv/runtime/state")); + assert_eq!( + nested.reservation("/srv/runtime"), + Some("/srv/runtime/state") + ); + assert_eq!(nested.reservation("/srv/other"), None); + // A root that keeps its whole tree takes no mapping. + let everything = ReservedTargets { + trees: &["/"], + exact: &[], + }; + for target in ["/", "/workspace", "/srv/runtime/state"] { + assert_eq!(everything.reservation(target), Some("/"), "{target}"); + } +} + +#[test] +fn policy_paths_hide_show_and_narrow_writes() { + let directory = root(); + let base = directory.path(); + fs::create_dir_all(base.join("work/src/secret/public")).unwrap(); + fs::create_dir_all(base.join("work/out/cache")).unwrap(); + let work = base.join("work"); + let paths = + |names: &[&str]| -> Vec { names.iter().map(|name| work.join(name)).collect() }; + let request = + |mappings: &[Mapping<'_>], denied: &[PathBuf], allowed: &[PathBuf], write: &[PathBuf]| { + let request = ExportRequest { + denied: PathList { + name: "denied", + paths: denied, + }, + allowed: PathList { + name: "allowed", + paths: allowed, + }, + write: PathList { + name: "write", + paths: write, + }, + ..ExportRequest::new(mappings, ReservedTargets::CONTAINER_ROOT) + }; + super::plan(&request) + }; + let whole = [at(&work, "/workspace", false)]; + let (denied, allowed, write) = ( + paths(&["src/secret"]), + paths(&["src/secret/public"]), + paths(&["out"]), + ); + let mapping = request(&whole, &denied, &allowed, &write).unwrap().unwrap(); + let child = &mapping.children[0]; + let resolved = canonical(&work); + assert!(child.writable); + assert_eq!(child.denied, [resolved.join("src").join("secret")]); + assert_eq!( + child.allowed, + [resolved.join("src").join("secret").join("public")] + ); + assert_eq!(child.write, [resolved.join("out")]); + // Writes are narrowed to `out`, so nothing outside it can move. + assert_eq!(child.denied_identities, [None]); + let arguments = mapping.openvmm_arguments(); + let text = |path: PathBuf| vec![display_path(&path)]; + assert_eq!( + option_values(&arguments, "--mount-deny"), + text(resolved.join("src").join("secret")) + ); + assert_eq!( + option_values(&arguments, "--mount-allow"), + text(resolved.join("src").join("secret").join("public")) + ); + assert_eq!( + option_values(&arguments, "--mount-write"), + text(resolved.join("out")) + ); + assert!(!arguments.contains(&"--mount-owner".to_owned())); + mapping.check(ReservedTargets::CONTAINER_ROOT).unwrap(); + + // A denied root shows only its allowed paths. + let mapping = request(&whole, std::slice::from_ref(&work), &paths(&["src"]), &[]) + .unwrap() + .unwrap(); + assert_eq!(mapping.children[0].denied, std::slice::from_ref(&resolved)); + assert_eq!(mapping.children[0].allowed, [resolved.join("src")]); + let error = request(&whole, std::slice::from_ref(&work), &[], &[]).unwrap_err(); + assert!( + error.message().contains("contains the mapped path"), + "{error}" + ); + + // A read-only path may lie in the part of a narrowed directory that stays read-only, and + // the directory may be mapped read-only elsewhere too. + let src = work.join("src"); + let parts = [ + at(&work, "/workspace", false), + at(&src, "/src", true), + at(&work, "/view", true), + ]; + request(&parts, &[], &[], &write).unwrap().unwrap(); + + let refused = |mappings: &[Mapping<'_>], + denied: &[PathBuf], + allowed: &[PathBuf], + write: &[PathBuf], + expected: &str| { + let error = request(mappings, denied, allowed, write).unwrap_err(); + assert_eq!(error.kind(), ErrorKind::Unsupported, "{error}"); + assert!( + error.message().contains(expected), + "{error} lacks {expected}" + ); + }; + refused( + &whole, + &[], + &paths(&["src"]), + &[], + "lies inside no denied entry", + ); + refused( + &whole, + std::slice::from_ref(&work), + std::slice::from_ref(&work), + &[], + "is an exported path itself", + ); + refused( + &whole, + &paths(&["src"]), + &paths(&["src/secret", "src/secret/public"]), + &[], + "which shows it already", + ); + refused( + &whole, + &paths(&["src"]), + &paths(&["src"]), + &[], + "is also a denied entry", + ); + refused( + &[at(&work, "/workspace", true)], + &[], + &[], + &write, + "lies in no directory that is mapped read-write as a whole", + ); + refused( + &whole, + &[], + &[], + std::slice::from_ref(&work), + "lies in no directory that is mapped read-write as a whole", + ); + refused( + &whole, + &paths(&["src"]), + &[], + &paths(&["src/secret"]), + "which hides it", + ); + let cache = work.join("out/cache"); + refused( + &[at(&work, "/workspace", false), at(&cache, "/cache", true)], + &[], + &[], + &write, + "lies inside the write entry", + ); + refused( + &[at(&work, "/workspace", false), at(&work, "/view", true)], + &[], + &[], + &[], + "names the object of the mappings entry", + ); +} + +#[test] +fn narrowed_writes_pin_only_what_lies_in_writable_paths() { + let directory = root(); + let base = directory.path(); + fs::create_dir_all(base.join("work/out/cache")).unwrap(); + fs::create_dir_all(base.join("work/out/private")).unwrap(); + let work = base.join("work"); + let (cache, src) = (work.join("out/cache"), work.join("src")); + let mappings = [ + at(&work, "/workspace", false), + at(&cache, "/cache", false), + at(&src, "/src", true), + ]; + let denied = [work.join("out/private"), work.join("src/secret")]; + let write = [work.join("out")]; + let request = ExportRequest { + denied: PathList { + name: "denied", + paths: &denied, + }, + write: PathList { + name: "write", + paths: &write, + }, + ..ExportRequest::new(&mappings, ReservedTargets::CONTAINER_ROOT) + }; + let mapping = super::plan(&request).unwrap().unwrap(); + let child = &mapping.children[0]; + assert_eq!(child.write, [canonical(&work).join("out")]); + let pinned: Vec = child + .denied_identities + .iter() + .map(Option::is_some) + .collect(); + assert_eq!(pinned, [true, false]); + let pinned: Vec<(&str, bool)> = mapping + .binds + .iter() + .map(|bind| (bind.target.as_str(), bind.identity.is_some())) + .collect(); + assert_eq!( + pinned, + [("/cache", true), ("/src", false), ("/workspace", false)] + ); + // A record keeps every pin of the plan, and narrows writes only to paths that keep its + // read-write mappings writable. + mapping.check(ReservedTargets::CONTAINER_ROOT).unwrap(); + let refused = |change: fn(&mut ExportPlan), expected: &str| { + let mut changed = mapping.clone(); + change(&mut changed); + let error = changed.check(ReservedTargets::CONTAINER_ROOT).unwrap_err(); + assert_eq!(error.kind(), ErrorKind::Unsupported, "{error}"); + assert!(error.message().contains(expected), "{error}"); + }; + refused( + |changed| changed.children[0].denied_identities[0] = None, + "does not pin the denied path", + ); + refused( + |changed| changed.binds[0].identity = None, + "does not pin the object that it maps at /cache", + ); + refused( + |changed| changed.children[0].write = vec![changed.children[0].root.join("other")], + "which it maps read-write, stays read-only", + ); + // Without write paths, the guest could move `src/secret` too. + refused( + |changed| changed.children[0].write.clear(), + "does not pin the denied path", + ); + mapping.verify().unwrap(); + fs::rename(&cache, work.join("out/moved")).unwrap(); + fs::create_dir(&cache).unwrap(); + assert_eq!(mapping.verify().unwrap_err().kind(), ErrorKind::Other); +} + +#[test] +fn the_caller_owner_reaches_openvmm_on_linux_hosts() { + let directory = root(); + let work = directory.path().join("work"); + let mappings = [at(&work, "/workspace", false)]; + let request = ExportRequest { + owner: Owner::Caller, + ..ExportRequest::new(&mappings, ReservedTargets::CONTAINER_ROOT) + }; + if cfg!(windows) { + let error = super::plan(&request).unwrap_err(); + assert!(error.message().contains("requires a Linux host"), "{error}"); + return; + } + #[cfg(unix)] + { + use std::os::unix::fs::MetadataExt; + + let owned = fs::metadata(&work).unwrap(); + if owned.uid() != 0 && owned.gid() != 0 { + let mapping = super::plan(&request).unwrap().unwrap(); + assert_eq!(mapping.owner, Owner::Caller); + let arguments = mapping.openvmm_arguments(); + assert_eq!( + arguments[arguments.len() - 2..], + ["--mount-owner", "caller"] + ); + mapping.verify().unwrap(); + let record = serde_json::to_value(&mapping).unwrap(); + assert_eq!(record["owner"], "caller"); + assert_eq!( + serde_json::from_value::(record).unwrap(), + mapping + ); + } + // Guest root is squashed to the owner, which must not be root. + let system = Path::new("/usr"); + if fs::metadata(system).is_ok_and(|metadata| metadata.uid() == 0) { + let mappings = [at(system, "/srv/system", true)]; + let request = ExportRequest { + owner: Owner::Caller, + ..ExportRequest::new(&mappings, ReservedTargets::CONTAINER_ROOT) + }; + let error = super::plan(&request).unwrap_err(); + assert!( + error + .message() + .contains("must not be owned by UID 0 or GID 0"), + "{error}" + ); + } + } + // The default owner is left out of the record. + let vmm = super::plan(&ExportRequest::new( + &mappings, + ReservedTargets::CONTAINER_ROOT, + )) + .unwrap() + .unwrap(); + assert!(serde_json::to_value(&vmm).unwrap().get("owner").is_none()); +} + +#[test] +fn read_write_mappings_to_which_writes_are_narrowed_stay_visible() { + let directory = root(); + let base = directory.path(); + fs::create_dir_all(base.join("tree/private/public")).unwrap(); + fs::create_dir_all(base.join("tree/out")).unwrap(); + let (tree, private) = (base.join("tree"), base.join("tree/private")); + let denied = [private.clone()]; + let allowed = [private.join("public")]; + let request = |mappings: &[Mapping<'_>], write: &[PathBuf]| { + super::plan(&ExportRequest { + denied: PathList { + name: "denied", + paths: &denied, + }, + allowed: PathList { + name: "allowed", + paths: &allowed, + }, + write: PathList { + name: "write", + paths: write, + }, + ..ExportRequest::new(mappings, ReservedTargets::CONTAINER_ROOT) + }) + }; + // OpenVMM narrows the writes of the tree to the read-write private, which it cannot hide: + // in a read-only tree, and in a read-write tree whose writes are narrowed. + let out = [tree.join("out")]; + for (mappings, write) in [ + ( + [at(&tree, "/tree", true), at(&private, "/private", false)], + &[][..], + ), + ( + [at(&tree, "/tree", false), at(&private, "/private", false)], + &out[..], + ), + ] { + let error = request(&mappings, write).unwrap_err(); + assert_eq!(error.kind(), ErrorKind::Unsupported, "{error}"); + assert!( + error + .message() + .contains("OpenVMM cannot make a hidden path writable"), + "{error}" + ); + } + // In a tree that is writable as a whole, the denied private shows its allowed paths. + let mapping = request( + &[at(&tree, "/tree", false), at(&private, "/private", false)], + &[], + ) + .unwrap() + .unwrap(); + let child = &mapping.children[0]; + assert!(child.writable && child.write.is_empty()); + assert_eq!(child.denied, [canonical(&private)]); + assert_eq!(child.allowed, [canonical(&private).join("public")]); +} + +#[test] +fn denied_mapped_directories_inside_denied_ones_show_their_allowed_paths() { + let directory = root(); + let base = directory.path(); + fs::create_dir_all(base.join("tree/private/public")).unwrap(); + fs::create_dir_all(base.join("tree/other")).unwrap(); + let (tree, private) = (base.join("tree"), base.join("tree/private")); + let denied = [tree.clone(), private.clone()]; + let request = |mappings: &[Mapping<'_>], allowed: &[PathBuf]| { + super::plan(&ExportRequest { + denied: PathList { + name: "denied", + paths: &denied, + }, + allowed: PathList { + name: "allowed", + paths: allowed, + }, + ..ExportRequest::new(mappings, ReservedTargets::CONTAINER_ROOT) + }) + }; + let both = [at(&tree, "/tree", false), at(&private, "/private", false)]; + // Each mapped directory is denied itself and shows the allowed path inside it; OpenVMM + // needs only the outer denial, which hides the inner one already. + let mapping = request(&both, &[private.join("public")]).unwrap().unwrap(); + let child = &mapping.children[0]; + assert_eq!(child.denied, [canonical(&tree)]); + assert_eq!(child.allowed, [canonical(&private).join("public")]); + mapping.check(ReservedTargets::CONTAINER_ROOT).unwrap(); + // OpenVMM receives each exported directory's policy alone, so one that a denied directory + // outside it contains keeps its own denial. + let mapping = request( + &[at(&private, "/private", false)], + &[private.join("public")], + ) + .unwrap() + .unwrap(); + let child = &mapping.children[0]; + assert_eq!(child.root, canonical(&private)); + assert_eq!(child.denied, [canonical(&private)]); + assert_eq!(child.allowed, [canonical(&private).join("public")]); + mapping.check(ReservedTargets::CONTAINER_ROOT).unwrap(); + // A mapped directory whose denial shows nothing again inside it stays hidden. + let error = request(&both, &[tree.join("other")]).unwrap_err(); + assert_eq!(error.kind(), ErrorKind::Unsupported, "{error}"); + assert!( + error.message().contains("contains the mapped path"), + "{error}" + ); +} + +#[test] +fn recorded_plans_hold_no_more_policy_than_one_aggregate_accepts() { + // Each child's paths of each kind stay within OpenVMM's limits for one directory. + let child = |index: usize| { + let root = std::env::temp_dir().join(format!("nvx-policy-{index}")); + let long = |kind: &str, path: usize| format!("{kind}-{path}-{}", "x".repeat(3990)); + let paths = |kind: &str| -> Vec { + (0..4).map(|path| root.join(long(kind, path))).collect() + }; + // OpenVMM shows paths again only inside a path that it hides. + let hidden = root.join(long("denied", 0)); + ExportChild { + root: root.clone(), + file: false, + writable: true, + denied: paths("denied"), + denied_identities: vec![None; 4], + allowed: (0..4) + .map(|path| hidden.join(format!("allowed-{path}"))) + .collect(), + write: paths("write"), + } + }; + let recorded = |count: usize| ExportPlan { + children: (0..count).map(child).collect(), + binds: (0..count) + .map(|index| Bind { + child: index, + source: String::new(), + target: format!("/c{index}"), + read_only: false, + identity: None, + }) + .collect(), + owner: Owner::Vmm, + }; + recorded(2).check(ReservedTargets::CONTAINER_ROOT).unwrap(); + let error = recorded(3) + .check(ReservedTargets::CONTAINER_ROOT) + .unwrap_err(); + assert_eq!(error.kind(), ErrorKind::Unsupported, "{error}"); + assert!( + error.message().contains("exceed its 131072-byte limit"), + "{error}" + ); +} + +#[cfg(target_os = "linux")] +#[test] +fn mapped_paths_resolve_to_utf8_paths() { + use std::ffi::OsStr; + use std::os::unix::ffi::OsStrExt; + + let directory = root(); + let base = directory.path(); + let share = base.join(OsStr::from_bytes(b"share-\xff")); + fs::create_dir(&share).unwrap(); + // A link of a valid name resolves to the same path, which OpenVMM's options could not + // name. + let link = base.join("link"); + std::os::unix::fs::symlink(&share, &link).unwrap(); + for path in [&share, &link] { + let mappings = [at(path, "/workspace", false)]; + let error = super::plan(&ExportRequest::new( + &mappings, + ReservedTargets::CONTAINER_ROOT, + )) + .unwrap_err(); + assert_eq!(error.kind(), ErrorKind::Unsupported, "{error}"); + assert!( + error + .message() + .contains("resolves to a path that is not valid UTF-8"), + "{error}" + ); + } +} diff --git a/nvx-rs/src/host/filesystem/verify.rs b/nvx-rs/src/host/filesystem/verify.rs new file mode 100644 index 000000000..c457be500 --- /dev/null +++ b/nvx-rs/src/host/filesystem/verify.rs @@ -0,0 +1,484 @@ +//! Checks and rendering of a recorded export plan. + +use std::fs; +use std::path::{Path, PathBuf}; + +use super::guest_path::depth; +use super::host_path::{display_path, relative_text}; +use super::limits::policy_limits; +use super::policy::{Writable, nearest_rule, policy_rule_problem}; +use super::{ + Bind, CALLER_OWNER_ON_WINDOWS, ExportChild, ExportPlan, FileIdentity, GUEST_EXPORT, + MAX_AGGREGATE_POLICY_BYTES, MAX_CHILDREN, MAX_GUEST_PATH, Owner, ReservedTargets, + check_caller_roots, outermost, resolved_path_problem, target_problem, volume_root_refusal, +}; +use crate::error::{Error, Result}; +use crate::host::platform; +use crate::protocol::{HostMap, MAPS_TOKEN, MAX_HOST_MAPPINGS}; + +impl ExportPlan { + /// Checks that a plan read back from a record is one that [`crate::host::filesystem::plan`] could have made for a root + /// that reserves `reserved`: at least one and at most [`MAX_CHILDREN`] children, none inside + /// another, each an absolute host path below a volume's root, with its host paths in the form + /// in which `plan` resolves them, whose policy paths lie inside it within OpenVMM's limits and + /// keep its rules, with an identity slot for each denied path, and writable exactly when a bind + /// maps a path of it read-write; and at least one and at most [`MAX_HOST_MAPPINGS`] binds, + /// each of a known child, from a relative source that holds the host's names joined with `/`, + /// that a guest path can hold, and that the child does not hide, unless it shows paths inside + /// it again, to a free guest target, parents before children, with a bind of every child's + /// root. A target inside another's must show what the other shows there. A child's write + /// paths must be the outermost paths of its read-write binds, unless one maps the whole child, + /// when they may narrow its writes only to paths that keep its other read-write binds + /// writable; and each denied or mapped object that a workload could move, strictly inside a + /// part of the child that the guest may write, must carry its identity. It reads nothing from + /// the host; [`verify`](Self::verify) checks the host. + /// + /// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported). + pub fn check(&self, reserved: ReservedTargets<'_>) -> Result<()> { + let invalid = |reason: String| { + Err(Error::unsupported(format!( + "the recorded export plan is invalid: {reason}" + ))) + }; + if self.children.is_empty() || self.children.len() > MAX_CHILDREN { + return invalid(format!( + "it exports {} directories and files, but OpenVMM exports from 1 to \ + {MAX_CHILDREN}", + self.children.len() + )); + } + if self.binds.is_empty() || self.binds.len() > MAX_HOST_MAPPINGS { + return invalid(format!( + "it maps {} paths, but a sandbox maps from 1 to {MAX_HOST_MAPPINGS}", + self.binds.len() + )); + } + let mut policy_bytes = 0; + for (index, child) in self.children.iter().enumerate() { + if !child.root.is_absolute() || volume_root_refusal(&child.root).is_some() { + return invalid(format!( + "child {index}, {}, is not an absolute path below a volume's root", + child.root.display() + )); + } + if let Some((path, problem)) = std::iter::once(&child.root) + .chain(&child.denied) + .chain(&child.allowed) + .chain(&child.write) + .find_map(|path| resolved_path_problem(path).map(|problem| (path, problem))) + { + return invalid(format!( + "child {index} has the host path {path:?}, which {problem}" + )); + } + if child.denied_identities.len() != child.denied.len() { + return invalid(format!( + "child {index} has {} denied paths but {} identities", + child.denied.len(), + child.denied_identities.len() + )); + } + if let Some(path) = [&child.denied, &child.allowed, &child.write] + .into_iter() + .flatten() + .find(|path| !path.starts_with(&child.root) || (child.file && *path != &child.root)) + { + return invalid(format!( + "child {index} has the policy path {}, which is not inside it", + path.display() + )); + } + for (paths, action) in [ + (&child.denied, "hide"), + (&child.allowed, "show again"), + (&child.write, "make writable"), + ] { + match policy_limits(&child.root, paths, action) { + Ok(bytes) => policy_bytes += bytes, + Err(problem) => return invalid(format!("child {index}: {problem}")), + } + } + if let Some(other) = self.children[..index].iter().position(|other| { + other.root.starts_with(&child.root) || child.root.starts_with(&other.root) + }) { + return invalid(format!( + "child {index}, {}, overlaps child {other}, {}", + child.root.display(), + self.children[other].root.display() + )); + } + if let Some(problem) = policy_rule_problem(child) { + return invalid(format!("child {index}: {problem}")); + } + if !child.writable && !child.write.is_empty() { + return invalid(format!("child {index} is read-only but has writable paths")); + } + if !self + .binds + .iter() + .any(|bind| bind.child == index && bind.source.is_empty()) + { + return invalid(format!("it maps no path at the root of child {index}")); + } + // OpenVMM lets the guest write a child that a read-write bind needs, and no other. + let read_write = self + .binds + .iter() + .any(|bind| bind.child == index && !bind.read_only); + if child.writable != read_write { + return invalid(if child.writable { + format!("child {index} is writable but maps no path read-write") + } else { + format!("child {index} is read-only but maps a path read-write") + }); + } + } + if policy_bytes > MAX_AGGREGATE_POLICY_BYTES { + return invalid(format!( + "the paths that OpenVMM is to hide, show again, or make writable exceed its \ + {MAX_AGGREGATE_POLICY_BYTES}-byte limit" + )); + } + let mut previous: Option<&Bind> = None; + for (index, bind) in self.binds.iter().enumerate() { + let Some(child) = self.children.get(bind.child) else { + return invalid(format!( + "it maps a path of the unknown child {}", + bind.child + )); + }; + if !bind.source.is_empty() + && (child.file + || bind + .source + .split('/') + .any(|component| matches!(component, "" | "." | ".."))) + { + return invalid(format!( + "it maps the path {:?} of child {}, which is not a path inside a directory", + bind.source, bind.child + )); + } + // The guest receives the source as a path below its mount of the export. + if bind.source.contains('\0') { + return invalid(format!( + "it maps the path {:?} of child {}, which contains a NUL character", + bind.source, bind.child + )); + } + if GUEST_EXPORT.len() + 1 + bind.guest_source().len() > MAX_GUEST_PATH { + return invalid(format!( + "it maps a path of {} bytes in child {}, which is too long for a guest path", + bind.source.len(), + bind.child + )); + } + let path = if bind.source.is_empty() { + child.root.clone() + } else { + child.root.join(&bind.source) + }; + // The host checks the object at that path, which must be the one that the guest maps: + // the source must hold the host's names below the root, joined with `/`, as `plan` + // spells them. A Windows host would read a backslash in a name as a separator, and a + // name with a root or a drive as another path. + if relative_text(&child.root, &path).as_deref() != Some(bind.source.as_str()) { + return invalid(format!( + "it maps the path {:?} of child {}, which the host reads as other names", + bind.source, bind.child + )); + } + // A mapped path shows something: OpenVMM does not hide it, or it shows the allowed + // paths inside it again. + if matches!( + nearest_rule(&path, &child.denied, &child.allowed, true), + Some((_, true)) + ) && !child + .allowed + .iter() + .any(|shown| *shown != path && shown.starts_with(&path)) + { + return invalid(format!( + "it maps {} at {}, but child {} hides it", + path.display(), + bind.target, + bind.child + )); + } + if let Some(problem) = target_problem(&bind.target) { + return invalid(format!("its guest target {:?} {problem}", bind.target)); + } + if let Some(reserved) = reserved.reservation(&bind.target) { + return invalid(format!( + "its guest target {} would cover the guest's {reserved}", + bind.target + )); + } + if let Some(previous) = previous + && (depth(&previous.target), previous.target.as_str()) + >= (depth(&bind.target), bind.target.as_str()) + { + return invalid(format!( + "it maps {} after {}, out of order or twice", + bind.target, previous.target + )); + } + // A target inside another shows what the other shows there, as `plan` requires. + for outer in &self.binds[..index] { + let Some(rest) = bind + .target + .strip_prefix(outer.target.as_str()) + .and_then(|rest| rest.strip_prefix('/')) + else { + continue; + }; + let shown = if outer.source.is_empty() { + rest.to_owned() + } else { + format!("{}/{rest}", outer.source) + }; + if bind.child != outer.child || bind.source != shown { + return invalid(format!( + "it maps {} inside {}, where it would hide part of that path", + bind.target, outer.target + )); + } + } + previous = Some(bind); + } + for (index, child) in self.children.iter().enumerate() { + let bound = |bind: &Bind| { + if bind.source.is_empty() { + child.root.clone() + } else { + child.root.join(&bind.source) + } + }; + let read_write: Vec = self + .binds + .iter() + .filter(|bind| bind.child == index && !bind.read_only) + .map(bound) + .collect(); + // A child that a read-write bind maps whole is writable throughout, unless write paths + // narrow its writes, which must then keep its other read-write binds writable; any + // other child's writes are narrowed to exactly what its read-write binds map. + if read_write.contains(&child.root) { + if let Some(path) = read_write.iter().find(|path| { + **path != child.root + && !child.write.is_empty() + && !child.write.iter().any(|write| path.starts_with(write)) + }) { + return invalid(format!( + "child {index} narrows its writes so that {}, which it maps read-write, \ + stays read-only", + path.display() + )); + } + } else { + let mut write = child.write.clone(); + write.sort(); + if outermost(read_write) != write { + return invalid(format!( + "child {index} lets the guest write other paths than those that it maps \ + read-write" + )); + } + } + // A workload that may write a part of the child could move an object there and put a + // decoy at its path, so the plan pins each denied and mapped object in such a part, as + // `plan` does, for `verify` to check it at the next start. + let part = Writable { + writable: child.writable, + write: child.write.clone(), + }; + if let Some((path, _)) = child + .denied + .iter() + .zip(&child.denied_identities) + .find(|(path, identity)| identity.is_none() && part.moves(&child.root, path)) + { + return invalid(format!( + "child {index} does not pin the denied path {}, which a workload could move", + path.display() + )); + } + if let Some(bind) = self.binds.iter().find(|bind| { + bind.child == index + && bind.identity.is_none() + && part.moves(&child.root, &bound(bind)) + }) { + return invalid(format!( + "it does not pin the object that it maps at {}, which a workload could move", + bind.target + )); + } + } + Ok(()) + } + + /// Checks that the host keeps every read-only mapping read-only, that every child's root is + /// still the kind of object that [`crate::host::filesystem::plan`] exported, that every mapped and denied path inside a + /// writable part still names the object it named then, and that the host still supports the + /// owner. + /// + /// [`crate::host::filesystem::plan`] refuses a read-only path inside a part that the guest may write, which only the + /// guest's bind mount would keep read-only; a recorded plan that holds one fails with + /// [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported). A file that a directory replaced + /// would otherwise grant the whole directory, and a workload that may write a directory could + /// rename a denied or mapped object inside it and put a decoy at its path, so the next start + /// would hide or bind the decoy instead; these fail with + /// [`ErrorKind::Other`](crate::ErrorKind::Other). With the caller owner, the host must be a + /// Linux host, and no exported directory or file may belong to UID 0 or GID 0; these fail with + /// [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported). Run it while the VM is stopped, + /// so that no workload can race with it; OpenVMM checks each child's kind again when it opens + /// it. + pub fn verify(&self) -> Result<()> { + for bind in self.binds.iter().filter(|bind| bind.read_only) { + let (child, path) = self.bound(bind)?; + // Without writable paths, the guest may write all of a writable child. + if child.writable + && (child.write.is_empty() + || child.write.iter().any(|write| path.starts_with(write))) + { + return Err(Error::unsupported(format!( + "the sandbox maps {} read-only inside a part of {} that the guest may write, \ + where the host cannot keep it read-only; deprovision the sandbox, and \ + provision it again without read-only paths inside read-write directories", + path.display(), + child.root.display() + ))); + } + } + let changed = |path: &Path| { + Err(Error::other(format!( + "{} no longer names the host object it named at provision; deprovision the \ + sandbox and provision it again", + path.display() + ))) + }; + for child in &self.children { + let kind_matches = fs::symlink_metadata(&child.root).is_ok_and(|metadata| { + if child.file { + metadata.is_file() + } else { + metadata.is_dir() + } + }); + if !kind_matches { + return changed(&child.root); + } + } + for bind in &self.binds { + let Some(expected) = bind.identity else { + continue; + }; + let (_, path) = self.bound(bind)?; + match platform::file_identity(&path) { + Ok((device, index)) if (FileIdentity { device, index }) == expected => {} + _ => return changed(&path), + } + } + for child in &self.children { + for (path, expected) in child.denied.iter().zip(&child.denied_identities) { + let Some(expected) = expected else { + continue; + }; + match platform::file_identity(path) { + Ok((device, index)) if (FileIdentity { device, index }) == *expected => {} + _ => return changed(path), + } + } + } + if self.owner == Owner::Caller { + if cfg!(windows) { + return Err(Error::unsupported(CALLER_OWNER_ON_WINDOWS)); + } + check_caller_roots(&self.children)?; + } + Ok(()) + } + + /// Returns the child that holds `bind` and the host path that `bind` maps. + fn bound(&self, bind: &Bind) -> Result<(&ExportChild, PathBuf)> { + let Some(child) = self.children.get(bind.child) else { + return Err(Error::io( + "the sandbox state maps a path of an unknown host directory", + )); + }; + let path = if bind.source.is_empty() { + child.root.clone() + } else { + child.root.join(&bind.source) + }; + Ok((child, path)) + } + + /// Kernel command-line token that tells the guest agent how many bind mounts to expect over + /// the control channel. + pub fn kernel_token(&self) -> String { + format!("{MAPS_TOKEN}{}", self.binds.len()) + } + + /// The bind mounts in the order and form in which the guest agent receives them. + pub fn guest_table(&self) -> Vec { + self.binds + .iter() + .map(|bind| HostMap { + source: bind.guest_source(), + target: bind.target.clone(), + read_only: bind.read_only, + }) + .collect() + } + + /// OpenVMM options that export the plan's directories and files at [`GUEST_EXPORT`], with + /// their access policies and the owner. A file child carries the `file` flag, so OpenVMM + /// refuses a directory that replaced the file. + pub fn openvmm_arguments(&self) -> Vec { + let mut arguments = vec!["--mount-aggregate".to_owned(), GUEST_EXPORT.to_owned()]; + for (index, child) in self.children.iter().enumerate() { + let mode = if child.writable { "rw" } else { "ro" }; + let kind = if child.file { ",file" } else { "" }; + arguments.push("--mount-child".to_owned()); + arguments.push(format!( + "{index},{},{mode}{kind}", + display_path(&child.root) + )); + } + for (option, paths) in [ + ( + "--mount-deny", + ExportChild::denied_paths as fn(&ExportChild) -> &[PathBuf], + ), + ("--mount-allow", ExportChild::allowed_paths), + ("--mount-write", ExportChild::write_paths), + ] { + for path in self.children.iter().flat_map(paths) { + arguments.push(option.to_owned()); + arguments.push(display_path(path)); + } + } + if self.owner == Owner::Caller { + arguments.extend(["--mount-owner".to_owned(), "caller".to_owned()]); + } + arguments + } +} + +impl ExportChild { + /// Returns the paths that OpenVMM hides. + fn denied_paths(&self) -> &[PathBuf] { + &self.denied + } + + /// Returns the paths that stay visible inside denied paths. + fn allowed_paths(&self) -> &[PathBuf] { + &self.allowed + } + + /// Returns the paths to which OpenVMM limits the guest's writes. + fn write_paths(&self) -> &[PathBuf] { + &self.write + } +} diff --git a/nvx-rs/src/host/instance.rs b/nvx-rs/src/host/instance.rs deleted file mode 100644 index 5a6ed6792..000000000 --- a/nvx-rs/src/host/instance.rs +++ /dev/null @@ -1,1255 +0,0 @@ -//! Supervision of one VM instance: its OpenVMM process, its control endpoint, and the records -//! that name them, through the [`store`](super::store), the [`supervisor`], and the control -//! [`session`]. -//! -//! An instance lives in a [`SandboxStore`] of its own: a directory private to the current user, -//! in which the store keeps the instance's lock, the journal of its launch, and the runtime -//! record of its VM, and in which the caller may keep files of its own under other names, which it -//! declares as [`CallerFiles`], such as the configuration record whose presence means that the -//! instance is provisioned: -//! -//! ```text -//! / -//! sandbox.lock the lock that serializes the instance's lifecycle transitions -//! launch.json the journal of a start in progress, or of one that was interrupted -//! runtime.json the runtime record of the running VM -//! control.capability launch capability (running only) -//! control.sock Linux control endpoint (running only) -//! openvmm.log OpenVMM output of the latest start -//! outcome.json OpenVMM outcome report of the latest run -//! ... the caller's files -//! ``` -//! -//! Lifecycle transitions run under the instance's lock, through the [`LockedInstance`] that -//! [`Instance::lock`] returns, so that every operation can run in a different process. The lock is -//! the store's [lifecycle lock](super::store#lifecycle-locks) of an existing sandbox, which knows -//! the caller's files: -//! -//! - [`LockedInstance::find`] finds the instance's VM as an orchestrator does after it restarts -//! within the same host boot, with [`supervisor::reattach`]: it ends a start whose caller died, -//! forgets a VM that exited, and compares the caller's runtime identity with the one that a -//! running VM was launched with. A VM of another runtime identity is [`Found::Mismatched`]: it -//! runs through artifacts or contracts that the caller does not have, so the instance does not -//! drive it, and only ends it. -//! - [`LockedInstance::start`] launches a detached OpenVMM process with the managed lifecycle -//! through [`supervisor::launch`], passes it a fresh 32-byte capability through standard input, -//! and waits until the guest agent answers on the authenticated control console. The agent must -//! also advertise the control features in [`GuestFeatures::REQUIRED`]; a guest image that lacks -//! one is terminated, because it would silently ignore the policy or request that needs the -//! feature. Start then hands the agent the table of mapped host paths, and fails if the agent -//! cannot mount them all. -//! - [`LockedInstance::stop`] asks the guest to shut down and falls back to terminating OpenVMM -//! through its verified process identity. -//! - [`LockedInstance::terminate`] stops a mismatched VM: it terminates OpenVMM through its -//! verified process identity, opens no control session, whose protocol the guest might not -//! speak, and reads no outcome report, which follows the other runtime's contracts. -//! - [`LockedInstance::remove`] removes a stopped instance, with the caller's files, and its -//! directory, through the store's [removal](super::store#removal). -//! -//! [`Instance::exec`] runs a workload through the control console of a [`Running`] VM without the -//! lock, and [`Execution::pump`] streams its output live. The guest serializes executions: a -//! second execution waits until the first finishes. Messages call the instance `sandbox NAME`, -//! after the name that [`Instance::create`] or [`Instance::open`] gives it. - -use std::fmt; -use std::fs::{self, OpenOptions}; -use std::io::{self, Write}; -use std::path::{Path, PathBuf}; -use std::sync::atomic::{AtomicBool, Ordering}; -use std::time::{Duration, Instant}; - -use serde::Serialize; -use serde::de::DeserializeOwned; - -use super::artifacts::Artifacts; -use super::launch::{LaunchSpec, timeout_problem}; -use super::platform; -use super::process::KILL_TIMEOUT; -use super::report::{self, Report}; -use super::session::{self, ControlSession, ExecEvent, SessionError}; -use super::store::{ - CallerFiles, IdentityPart, LaunchId, LockedStore, OUTCOME_NAME, Recovery, RemovalError, - RuntimeIdentity, RuntimeRecord, SOCKET_NAME, SandboxStore, -}; -use super::supervisor::{self, Reattached, Starting, StopOutcome, Vmm, VmmCommand}; -use crate::error::{Error, ErrorCode, ErrorKind, Result}; -use crate::protocol::{ - CAPABILITY_LEN, CWD_FAILED, ExitCategory, GuestFeatures, LAUNCH_FAILED, MAPPINGS_INCOMPLETE, - MAX_OUTPUT_BYTES, Workload, -}; - -/// Longest delay between a cancellation request and its delivery to the guest. -const CANCEL_POLL_INTERVAL: Duration = Duration::from_millis(20); - -/// Builds an error mapper that preserves an I/O source under `context`. -fn io_error(context: String) -> impl FnOnce(io::Error) -> Error { - move |error| Error::io(context).with_source(error) -} - -/// Returns create-and-write options that request owner-only permissions on Unix. -fn private_options() -> OpenOptions { - let mut options = OpenOptions::new(); - options.write(true).create(true); - #[cfg(unix)] - std::os::unix::fs::OpenOptionsExt::mode(&mut options, 0o600); - options -} - -/// Returns the name of the temporary file through which [`write_private`] replaces the file `name` -/// for the random `number`: `.NAME.NUMBER.tmp`, which stays beside it if the replacement is -/// interrupted. -fn temporary_name(name: &str, number: u64) -> String { - format!(".{name}.{number}.tmp") -} - -/// Atomically replaces `path` with `contents`, readable only by the current user, as callers do -/// with the configuration records that they keep beside an instance's store. -/// -/// The contents go to a temporary file beside `path` first, which a replacement that is -/// interrupted leaves, and which [`remove_interrupted_writes`] removes. -pub fn write_private(path: &Path, contents: &[u8]) -> Result<()> { - let mut suffix = [0u8; 8]; - getrandom::fill(&mut suffix) - .map_err(|error| Error::other("cannot name a temporary file").with_source(error))?; - let name = path - .file_name() - .map(|name| name.to_string_lossy().into_owned()) - .unwrap_or_default(); - let temporary = path.with_file_name(temporary_name(&name, u64::from_le_bytes(suffix))); - let written = (|| -> io::Result<()> { - let mut file = private_options().create_new(true).open(&temporary)?; - file.write_all(contents)?; - file.sync_all()?; - drop(file); - fs::rename(&temporary, path) - })(); - if written.is_err() { - let _ = fs::remove_file(&temporary); - } - written.map_err(io_error(format!("cannot write {}", path.display()))) -} - -/// Atomically replaces `path` with `value` as pretty JSON and a newline, readable only by the -/// current user. -pub fn write_json(path: &Path, value: &T) -> Result<()> { - let mut text = serde_json::to_vec_pretty(value) - .map_err(|error| Error::other("cannot encode sandbox state").with_source(error))?; - text.push(b'\n'); - write_private(path, &text) -} - -/// Reads the JSON at `path`, or returns `None` if the file does not exist. -pub fn read_json(path: &Path) -> Result> { - let text = match fs::read(path) { - Ok(text) => text, - Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(None), - Err(error) => return Err(io_error(format!("cannot read {}", path.display()))(error)), - }; - serde_json::from_slice(&text).map(Some).map_err(|error| { - Error::io(format!("sandbox state {} is malformed", path.display())).with_source(error) - }) -} - -/// Removes the file at `path` if it exists. -pub fn remove_if_present(path: &Path) -> Result<()> { - match fs::remove_file(path) { - Err(error) if error.kind() != io::ErrorKind::NotFound => { - Err(io_error(format!("cannot remove {}", path.display()))(error)) - } - _ => Ok(()), - } -} - -/// Removes the temporary files that interrupted replacements of `path` with [`write_private`] or -/// [`write_json`] left beside it, and keeps every other file. A directory that does not exist has -/// none. -/// -/// The store's removal refuses those files, which are neither the store's nor the files that its -/// caller declares, so a caller that writes its files with [`write_private`] removes them first, -/// under the instance's lock. -pub fn remove_interrupted_writes(path: &Path) -> Result<()> { - let Some(name) = path.file_name().map(|name| name.to_string_lossy()) else { - return Ok(()); - }; - let dir = match path.parent() { - Some(dir) if !dir.as_os_str().is_empty() => dir, - _ => Path::new("."), - }; - let unlisted = || io_error(format!("cannot list {}", dir.display())); - let entries = match fs::read_dir(dir) { - Ok(entries) => entries, - Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(()), - Err(error) => return Err(unlisted()(error)), - }; - for entry in entries { - let entry = entry.map_err(unlisted())?; - if entry - .file_name() - .to_str() - .is_some_and(|entry| is_temporary_of(entry, &name)) - { - remove_if_present(&entry.path())?; - } - } - Ok(()) -} - -/// Returns whether `entry` is the name of a temporary file through which [`write_private`] -/// replaces the file `name`. -fn is_temporary_of(entry: &str, name: &str) -> bool { - entry - .strip_suffix(".tmp") - .and_then(|rest| rest.rsplit_once('.')) - .and_then(|(_, number)| number.parse::().ok()) - .is_some_and(|number| temporary_name(name, number) == entry) -} - -/// Returns a fresh launch capability, which is never all zero. -pub fn generate_capability() -> Result<[u8; CAPABILITY_LEN]> { - let mut capability = [0u8; CAPABILITY_LEN]; - while capability == [0; CAPABILITY_LEN] { - getrandom::fill(&mut capability).map_err(|error| { - Error::other("cannot generate a control capability").with_source(error) - })?; - } - Ok(capability) -} - -/// Returns a control endpoint of an instance whose Linux control socket is `socket`: the socket -/// on Linux, and a fresh named pipe, whose name always has the same length, on Windows, as a -/// launch chooses it. -fn control_endpoint(socket: &Path) -> Result { - platform::control_endpoint(socket) - .map_err(|error| Error::io("cannot choose a control endpoint").with_source(error)) -} - -/// Fails with [`ErrorKind::Unsupported`] if OpenVMM's command line for `spec`, launched for an -/// instance in the directory `dir`, would not fit the limit of a Windows process command line; -/// see [`LaunchSpec::check_process_command_line`]. -/// -/// The directory need not exist yet, so that a caller can check a launch before it creates the -/// instance. Every control endpoint of an instance has the same length. -pub fn check_launch(dir: &Path, spec: &LaunchSpec) -> Result<()> { - let dir = std::path::absolute(dir).map_err(io_error(format!( - "cannot resolve the instance directory {}", - dir.display() - )))?; - let endpoint = control_endpoint(&dir.join(SOCKET_NAME))?; - spec.check_process_command_line(&endpoint, &dir.join(OUTCOME_NAME)) -} - -/// Classifies a session failure. -fn session_kind(error: &SessionError) -> ErrorKind { - match error { - SessionError::TimedOut => ErrorKind::TimedOut, - SessionError::ProcessExited => ErrorKind::Exited, - SessionError::Refused(_) | SessionError::Io(_) => ErrorKind::Io, - SessionError::Closed - | SessionError::Reset - | SessionError::AuthenticationFailed - | SessionError::Protocol(_) => ErrorKind::Protocol, - } -} - -/// Returns the code of a session failure: [`ErrorCode::EndpointClosed`] for a session that the -/// control endpoint closed, as OpenVMM closes it when the VM shuts down, and otherwise the code of -/// its [kind](session_kind). -fn session_code(error: &SessionError) -> ErrorCode { - match error { - SessionError::Closed => ErrorCode::EndpointClosed, - other => session_kind(other).into(), - } -} - -/// Explains why a launch did not become ready. -fn readiness_failure(error: &SessionError, runs_as_host_user: bool) -> String { - let mut reason = error.to_string(); - // An image that predates account creation ends its boot, before the guest agent starts, when - // it finds no account for the host user that the workloads run as. The VM then vanishes, or - // its control endpoint closes first. - let vm_ended = matches!(error, SessionError::ProcessExited | SessionError::Closed); - if runs_as_host_user && vm_ended { - reason.push_str( - "; a guest image that predates workload account creation exits during boot when \ - workloads run under the host user's IDs, so use an NVX release or build that \ - includes it", - ); - } - reason -} - -/// A VM that [`LockedInstance::start`] launched and the guest agent of which is ready. -#[derive(Debug, Clone, PartialEq, Eq)] -pub struct Started { - /// The runtime record of its launch: its OpenVMM process, its control endpoint, and the - /// runtime identity that it was launched with. - pub runtime: RuntimeRecord, - /// Time from launch to guest readiness. - pub boot: Duration, -} - -/// A VM that [`LockedInstance::stop`] stopped, or that [`LockedInstance::terminate`] ended. -#[derive(Debug, Clone, PartialEq, Eq)] -pub struct Stopped { - /// Whether OpenVMM was terminated: because the guest did not shut down in time, or because - /// the VM ran under another runtime identity. - pub forced: bool, - /// Summary of OpenVMM's outcome report, if it wrote one. [`LockedInstance::terminate`] reads - /// none, since the other runtime's OpenVMM wrote it. - pub report: Option, -} - -/// The running VM of an instance, which [`LockedInstance::find`] found under the caller's runtime -/// identity, and the capability with which its control sessions attach. -pub struct Running { - /// The VM, held through its verified process identity. - vmm: Vmm, - /// The launch capability of the VM, or the error with which reading it failed: only the - /// VM's control sessions need it, and a stop ends the VM without one. - capability: Result<[u8; CAPABILITY_LEN]>, -} - -impl Running { - /// Returns the VM, held through its verified process identity. - pub fn vmm(&self) -> &Vmm { - &self.vmm - } - - /// Returns the runtime record of the VM's launch. - pub fn runtime(&self) -> &RuntimeRecord { - self.vmm.record() - } - - /// Returns the launch capability of the VM, or fails as reading it failed, with the same - /// code, description, and the description of its cause. - fn capability(&self) -> Result<&[u8; CAPABILITY_LEN]> { - self.capability.as_ref().map_err(|error| { - let copy = Error::new(error.code(), error.message()); - match std::error::Error::source(error) { - Some(cause) => copy.with_source(io::Error::other(cause.to_string())), - None => copy, - } - }) - } -} - -impl fmt::Debug for Running { - fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { - // The capability authenticates control sessions, so it is never shown. - formatter - .debug_struct("Running") - .field("vmm", &self.vmm) - .finish_non_exhaustive() - } -} - -/// The running VM of an instance, which [`LockedInstance::find`] found under another runtime -/// identity than the caller's, and which [`LockedInstance::terminate`] ends. -#[derive(Debug)] -pub struct Mismatched { - /// The instance's name in messages. - name: String, - /// The VM, held through its verified process identity. - vmm: Vmm, - /// The parts of the runtime identity that differ. - differences: Vec, -} - -impl Mismatched { - /// Returns the VM, held through its verified process identity. - pub fn vmm(&self) -> &Vmm { - &self.vmm - } - - /// Returns the parts of the runtime identity that differ, as - /// [`RuntimeIdentity::differences`] returns them. - pub fn differences(&self) -> &[IdentityPart] { - &self.differences - } - - /// Returns the error that reports the VM, which the instance does not drive, since only the - /// build and artifacts that started it speak its contracts: it names the parts that differ, - /// and that stopping the VM terminates it. Its code is [`ErrorCode::RuntimeMismatch`]. - pub fn error(&self) -> Error { - let parts: Vec<&str> = self.differences.iter().map(|part| part.name()).collect(); - Error::new( - ErrorCode::RuntimeMismatch, - format!( - "sandbox {} runs under another runtime identity, which differs in {}, so it is \ - not driven: stop it to terminate its OpenVMM process {} without a guest shutdown", - self.name, - parts.join(", "), - self.vmm.record().process().pid - ), - ) - } -} - -/// What [`LockedInstance::find`] found of an instance's VM. -#[derive(Debug)] -pub enum Found { - /// No VM of the instance runs: none was started, the last one stopped or exited, or a start - /// that was interrupted is over. - Stopped, - /// The VM runs under the caller's runtime identity. - Running(Running), - /// The VM runs under another runtime identity: the caller ends it with - /// [`LockedInstance::terminate`], or reports it, as [`Mismatched::error`] does. - Mismatched(Mismatched), -} - -/// One VM instance in a sandbox store of its own. -#[derive(Debug, Clone, PartialEq, Eq)] -pub struct Instance { - /// The store of the instance's runtime records, logs, and control endpoint. - store: SandboxStore, - /// Sandbox name used in diagnostics. - name: String, - /// The files that the caller keeps in the instance's directory. - files: CallerFiles, -} - -impl Instance { - /// Creates the directory `dir` of a new instance, whose parent must exist, private to the - /// current user, as [`SandboxStore::create`] does, and supervises the instance in it, which - /// messages call `sandbox NAME`, and in which the caller keeps `files`. - pub fn create( - dir: impl Into, - name: impl Into, - files: CallerFiles, - ) -> Result { - Ok(Self { - store: SandboxStore::create(dir)?, - name: name.into(), - files, - }) - } - - /// Supervises the instance in the existing directory `dir`, which must be private to the - /// current user, as [`SandboxStore::open`] requires, which messages call `sandbox NAME`, and - /// in which the caller keeps `files`. Fails with [`ErrorKind::NotFound`] if `dir` does not - /// exist. - pub fn open( - dir: impl Into, - name: impl Into, - files: CallerFiles, - ) -> Result { - Ok(Self { - store: SandboxStore::open(dir)?, - name: name.into(), - files, - }) - } - - /// Returns the store of the instance's runtime records. - pub fn store(&self) -> &SandboxStore { - &self.store - } - - /// Returns the instance's directory, as an absolute path. - pub fn dir(&self) -> &Path { - self.store.dir() - } - - /// Returns the instance's name in messages. - pub fn name(&self) -> &str { - &self.name - } - - /// Returns the files that the caller keeps in the instance's directory. - pub fn files(&self) -> &CallerFiles { - &self.files - } - - /// Returns the path of the OpenVMM log of the latest start. - pub fn log_path(&self) -> PathBuf { - self.store.log_path() - } - - /// Returns the path of OpenVMM's outcome report of the latest run. - pub fn report_path(&self) -> PathBuf { - self.store.outcome_path() - } - - /// Takes the instance's lock, which serializes its lifecycle transitions, waiting for it - /// until `deadline`, or for as long as it takes without one, as - /// [`SandboxStore::lock_existing`] does with the caller's files. - /// - /// Fails with [`ErrorKind::NotFound`] once the instance's directory is gone, as a removal - /// that this waited for leaves it, and with [`ErrorKind::TimedOut`] when the deadline passes - /// first. A caller that keeps a configuration record checks it under the lock: the lock of a - /// directory without it is released without a lock file. - pub fn lock(&self, deadline: Option) -> Result { - Ok(LockedInstance { - locked: SandboxStore::lock_existing(self.dir(), &self.files, deadline)?, - instance: self.clone(), - }) - } - - /// Describes why this instance did not start and points the caller to its log. - fn start_failure(&self, code: impl Into, reason: &str) -> Error { - Error::new( - code, - format!( - "sandbox {} failed to start: {reason}; see {}", - self.name, - self.log_path().display() - ), - ) - } - - /// Ends a launch that did not become ready, as [`Starting::abandon`] does, and describes why - /// the instance did not start. - /// - /// A launch whose OpenVMM cannot be ended keeps its journal, and the next transition of the - /// instance ends it, when [`LockedInstance::find`] recovers it. - fn abort_start( - &self, - starting: Starting<'_>, - code: impl Into, - reason: &str, - ) -> Error { - let code = code.into(); - let unended = match starting.abandon() { - Ok(Recovery::Indeterminate(unended)) => unended, - Ok(_) => return self.start_failure(code, reason), - Err(error) => error.to_string(), - }; - self.start_failure( - code, - &format!( - "{reason}; the launch could not be ended, and remains for recovery: {unended}" - ), - ) - } - - /// Ends a launch whose guest image lacks control features that NVX needs. - /// - /// An older image would boot and answer, but it would silently ignore host path mappings and - /// cancellation requests, so the VM must not run at all. - fn reject_guest(&self, starting: Starting<'_>, missing: &[&str]) -> Error { - let reason = format!( - "the guest image lacks control features that the openvmm backend needs ({}); use an \ - NVX release or build that includes them, for example through {}", - missing.join(", "), - Artifacts::ENV_DIR - ); - self.abort_start(starting, ErrorCode::GuestFeaturesMissing, &reason) - } - - /// Maps a control-session failure onto an instance-facing error and message. - fn session_error(&self, error: SessionError, control_timeout: Duration) -> Error { - match error { - SessionError::ProcessExited => Error::new( - ErrorCode::SandboxExited, - format!("sandbox {} stopped unexpectedly", self.name), - ), - SessionError::TimedOut => Error::timed_out(format!( - "sandbox {} did not accept a control session within {} s; another exec may \ - still be running", - self.name, - control_timeout.as_secs() - )), - other => Error::new( - session_code(&other), - format!( - "the control session with sandbox {} failed: {other}", - self.name - ), - ) - .with_source(other), - } - } - - /// Starts `workload` in the instance's running VM, `running`, without the instance's lock. - /// - /// It waits up to `control_timeout` for the control session, which the guest grants one - /// execution at a time. [`Execution::pump`] then waits for the outcome up to the workload's - /// own timeout plus `response_grace`, and without a limit when the workload has no timeout. - /// Fails with [`ErrorKind::Exited`], and the code [`ErrorCode::SandboxExited`], when the VM is - /// gone, and with [`ErrorKind::Unsupported`] when `control_timeout` is zero or either - /// duration is longer than [`MAX_TIMEOUT`](super::launch::MAX_TIMEOUT). Fails as reading the - /// capability failed when [`find`](LockedInstance::find) could not read it. - pub fn exec( - &self, - running: &Running, - workload: &Workload<'_>, - control_timeout: Duration, - response_grace: Duration, - ) -> Result { - check_timeout("control timeout", control_timeout, true)?; - check_timeout("response grace", response_grace, false)?; - let capability = running.capability()?; - let deadline = Instant::now() + control_timeout; - let process = running.vmm.record().process(); - let mut session = session::connect( - running.vmm.endpoint(), - process.pid, - process.start_time, - deadline, - ) - .and_then(|transport| ControlSession::attach(transport, capability, deadline)) - .map_err(|error| self.session_error(error, control_timeout))?; - let request_id = session - .start_exec(workload, deadline) - .map_err(|error| self.session_error(error, control_timeout))?; - let timeout_ms = workload.timeout_ms; - let response_deadline = (timeout_ms > 0) - .then(|| Instant::now() + Duration::from_millis(timeout_ms.into()) + response_grace); - Ok(Execution { - session, - request_id, - deadline: response_deadline, - control_timeout, - }) - } -} - -/// An instance whose lock this holds, through which its lifecycle transitions change its -/// records. Dropping it releases the lock. -#[derive(Debug)] -pub struct LockedInstance { - /// The instance. - instance: Instance, - /// Its locked store. - locked: LockedStore, -} - -impl LockedInstance { - /// Returns the instance. - pub fn instance(&self) -> &Instance { - &self.instance - } - - /// Finds the instance's VM with [`supervisor::reattach`], under the runtime identity that - /// `runtime` returns: the one with which the caller would launch it now. - /// - /// `runtime` is asked only when the store records a VM or a start, so that an instance that - /// never started, or that stopped, needs no runtime identity. Reattach ends a start that was - /// interrupted, signaling only an OpenVMM process that it verified, and forgets a VM that - /// exited, as one that a record of an earlier host boot names has. Fails as `runtime` fails, - /// for records that the store does not read, such as those of an earlier format, whose - /// message names it, and when whether the VM runs cannot be determined, because an - /// interrupted start cannot be ended yet or the recorded process cannot be examined: the - /// records then remain for a later attempt. - /// - /// A VM whose capability cannot be read, as when its file is missing or damaged, still runs, - /// and is found [`Found::Running`]: [`Instance::exec`] then fails as reading it failed, and - /// [`stop`](Self::stop) ends the VM through its verified process identity. - pub fn find(&mut self, runtime: impl FnOnce() -> Result) -> Result { - if self.locked.journal()?.is_none() && self.locked.runtime()?.is_none() { - return Ok(Found::Stopped); - } - let runtime = runtime()?; - Ok(match supervisor::reattach(&mut self.locked, &runtime)? { - Reattached::Live(vmm) => Found::Running(Running { - vmm, - capability: self.locked.read_capability(), - }), - Reattached::Mismatched { vmm, differences } => Found::Mismatched(Mismatched { - name: self.instance.name.clone(), - vmm, - differences, - }), - Reattached::Exited { .. } => Found::Stopped, - Reattached::Indeterminate(reason) => { - return Err(Error::new( - ErrorCode::VmmIndeterminate, - format!( - "cannot determine whether the OpenVMM process of sandbox {} runs: \ - {reason}", - self.instance.name - ), - )); - } - }) - } - - /// Launches the VM that `spec` describes under the runtime identity `runtime`, and waits up - /// to `timeout` until its guest is ready. - /// - /// The caller has found the instance stopped with [`find`](Self::find). Start first checks the - /// export plan with [`ExportPlan::verify`](super::filesystem::ExportPlan::verify): that the - /// host keeps its read-only paths read-only, and that the mapped host paths still name the - /// objects they named when they were planned. It launches OpenVMM with - /// [`supervisor::launch`], whose journal records the launch before OpenVMM is spawned and its - /// process identity right after, so that [`find`](Self::find) ends a start whose caller died. - /// The capability travels through OpenVMM's standard-input pipe, which is closed before - /// OpenVMM starts, and never appears in arguments, the environment, or the log. Once the guest - /// is ready, has the required features, and has mounted the mapped host paths, start commits - /// the launch: the runtime record names the VM from then on. - /// - /// Fails with [`ErrorKind::Unavailable`], and the code [`ErrorCode::GuestFeaturesMissing`], - /// when the guest image lacks a feature in [`GuestFeatures::REQUIRED`], and with - /// [`ErrorKind::Unsupported`] when `timeout` is zero or - /// longer than [`MAX_TIMEOUT`](super::launch::MAX_TIMEOUT), when the export plan maps a - /// read-only path that the host cannot keep read-only, when the specification fails - /// [`LaunchSpec::check`] or exceeds a limit, or when OpenVMM's command line would not fit - /// [`PROCESS_COMMAND_LINE_LIMIT`](super::launch::PROCESS_COMMAND_LINE_LIMIT), which holds on - /// every host; such a launch records nothing. Unlike [`check_launch`], start keeps no reserve. - /// A launch that does not become ready is terminated through its verified process identity; - /// one that cannot be ended keeps its journal, and the next transition ends it. - pub fn start( - &mut self, - spec: &LaunchSpec, - runtime: &RuntimeIdentity, - timeout: Duration, - ) -> Result { - check_timeout("start timeout", timeout, true)?; - if let Some(plan) = &spec.filesystem { - plan.verify()?; - } - let instance = &self.instance; - let report = instance.report_path(); - // Every control endpoint of the instance has the same length, so a launch that would not - // fit is refused before the store records it. - let endpoint = control_endpoint(&instance.store.socket_path())?; - spec.fit_process_command_line(&spec.openvmm_arguments(&endpoint, &report)?, 0)?; - let capability = generate_capability()?; - let launch = LaunchId::generate()?; - let started = Instant::now(); - let starting = supervisor::launch( - &mut self.locked, - &launch, - runtime, - &capability, - |endpoint| { - let arguments = spec.openvmm_arguments(endpoint, &report)?; - spec.fit_process_command_line(&arguments, 0)?; - let mut command = VmmCommand::new(&spec.openvmm, arguments, instance.dir()); - command.breakaway_from_job = spec.breakaway_from_job; - Ok(command) - }, - )?; - let (pid, start_time) = { - let reference = starting.process().reference(); - (reference.pid, reference.start_time) - }; - let deadline = started + timeout; - let ready = session::connect(starting.endpoint(), pid, start_time, deadline) - .and_then(|transport| ControlSession::attach(transport, &capability, deadline)) - .and_then(|mut session| { - session.ping(deadline)?; - let features = session.features(deadline)?; - Ok((session, features)) - }); - let (mut session, features) = match ready { - Ok(ready) => ready, - Err(error) => { - let reason = readiness_failure(&error, spec.identity.create_account); - return Err(instance.abort_start(starting, session_code(&error), &reason)); - } - }; - let missing = features.missing(GuestFeatures::REQUIRED); - if !missing.is_empty() { - drop(session); - return Err(instance.reject_guest(starting, &missing)); - } - // Workloads run only once the guest has mounted every mapped host path. - if let Some(plan) = &spec.filesystem - && let Err(error) = session.map_host_paths(&plan.guest_table(), deadline) - { - drop(session); - let reason = format!("the host paths were not mapped: {error}"); - return Err(instance.abort_start(starting, session_code(&error), &reason)); - } - drop(session); - let vmm = starting.commit()?; - Ok(Started { - runtime: vmm.record().clone(), - boot: started.elapsed(), - }) - } - - /// Asks the guest of the running VM `running` to shut down, and terminates OpenVMM through - /// its verified process identity if it has not exited within `timeout`, as [`Vmm::stop`] - /// does with a forced step, or at once when the guest cannot be asked, as without the VM's - /// capability. - /// - /// A running execution keeps the control session until it ends, so a stop that cannot reach - /// the guest in time terminates the VM, and the execution then fails. Once OpenVMM has - /// exited, the stop clears the runtime record and reads OpenVMM's outcome report. Fails with - /// [`ErrorKind::Unsupported`] when `timeout` is zero or longer than - /// [`MAX_TIMEOUT`](super::launch::MAX_TIMEOUT), and when OpenVMM cannot be terminated, or does - /// not exit after it was, which keeps its runtime record. - pub fn stop(&mut self, running: Running, timeout: Duration) -> Result { - check_timeout("stop timeout", timeout, true)?; - let deadline = Instant::now() + timeout; - let Running { vmm, capability } = running; - let stopped = vmm.stop( - &mut self.locked, - deadline, - Some(KILL_TIMEOUT), - |vmm, deadline| { - // Without its capability, no session can reach the guest, and the forced step - // ends the VM. - let capability = - capability.map_err(|error| SessionError::Protocol(error.to_string()))?; - let process = vmm.record().process(); - let transport = - session::connect(vmm.endpoint(), process.pid, process.start_time, deadline)?; - ControlSession::attach(transport, &capability, deadline)?.stop(deadline) - }, - )?; - let forced = match stopped.outcome { - StopOutcome::Graceful => false, - StopOutcome::Forced => true, - StopOutcome::TimedOut => { - return Err(Error::other(format!( - "OpenVMM process {} of sandbox {} did not exit after it was terminated", - vmm.record().process().pid, - self.instance.name - ))); - } - }; - Ok(Stopped { - forced, - report: report::read(&self.instance.report_path()), - }) - } - - /// Stops the VM `mismatched`, which runs under another runtime identity than the caller's, by - /// identity-verified forced termination: terminates OpenVMM through its verified process - /// identity, as [`Vmm::terminate`] does, and waits up to `timeout` for it to exit. - /// - /// The VM's guest might not speak this build's control protocol, so the stop opens no control - /// session to it, and does not ask it to shut down: only the OpenVMM process that - /// [`find`](Self::find) verified is signaled. Once OpenVMM has exited, the stop clears the - /// runtime record. It reads no outcome report, so [`Stopped::report`] is `None`: a terminated - /// OpenVMM writes none, and one that exited before it was terminated wrote it under the other - /// runtime's contracts. [`Stopped::forced`] is false only for an OpenVMM that exited before it - /// was terminated. - /// - /// Fails with [`ErrorKind::Unsupported`] when `timeout` is zero or longer than - /// [`MAX_TIMEOUT`](super::launch::MAX_TIMEOUT), and when OpenVMM's state cannot be - /// determined, OpenVMM cannot be terminated, or it does not exit within `timeout`: its runtime - /// record then remains, and a later stop retries. - pub fn terminate(&mut self, mismatched: Mismatched, timeout: Duration) -> Result { - check_timeout("stop timeout", timeout, true)?; - let Mismatched { vmm, .. } = mismatched; - let forced = match vmm.terminate(&mut self.locked, timeout)? { - StopOutcome::Graceful => false, - StopOutcome::Forced => true, - StopOutcome::TimedOut => { - return Err(Error::other(format!( - "OpenVMM process {} of sandbox {} did not exit after it was terminated", - vmm.record().process().pid, - self.instance.name - ))); - } - }; - Ok(Stopped { - forced, - report: None, - }) - } - - /// Removes the stopped instance: its store's records and files, the caller's files that it - /// declared, the configuration record last, and its directory, through the store's - /// [`LockedStore::remove`], and releases the lock. Open files, such as the lock files of other - /// transitions, that keep the emptied directory are then waited for until `deadline`, or for - /// as long as it takes without one, as [`Lingering::wait`](super::store::Lingering::wait) - /// does. - /// - /// The caller has found the instance stopped with [`find`](Self::find): the removal refuses, - /// and removes nothing, while the store records a VM or a start, and for an instance whose - /// caller declared no configuration record. Entries in the directory that are neither the - /// store's nor the caller's are refused, and kept with the directory, once the rest is gone, - /// among them the temporary files that interrupted writes of the caller's files left, which - /// the caller removes first with [`remove_interrupted_writes`]. - pub fn remove(self, deadline: Option) -> std::result::Result<(), RemovalError> { - match self.locked.remove()? { - None => Ok(()), - Some(mut lingering) => lingering.wait(deadline), - } - } -} - -/// Fails with [`ErrorKind::Unsupported`] when the duration that `name` describes is longer than -/// [`MAX_TIMEOUT`](super::launch::MAX_TIMEOUT), or zero while it must be `positive`, so that no -/// deadline derived from it overflows. -fn check_timeout(name: &str, value: Duration, positive: bool) -> Result<()> { - match timeout_problem(value, positive) { - Some(problem) => Err(Error::unsupported(format!("the {name} {problem}"))), - None => Ok(()), - } -} - -/// Receives a workload's output. -pub trait OutputSink { - /// Writes one chunk. After an error, the chunks of this stream are discarded, and the - /// workload continues. - fn write(&mut self, chunk: &[u8]) -> io::Result<()>; -} - -impl io::Result<()>> OutputSink for F { - fn write(&mut self, chunk: &[u8]) -> io::Result<()> { - self(chunk) - } -} - -/// How a workload ended. -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -pub enum WorkloadOutcome { - /// The workload exited with this code. - Exited(i32), - /// A signal with this number ended the workload. - Signaled(i32), - /// The workload's timeout elapsed, and the guest agent killed it. - TimedOut, - /// The workload exceeded [`MAX_OUTPUT_BYTES`], and the guest agent killed it. - OutputLimitExceeded, - /// The guest agent could not supervise the workload. - Failed, - /// A cancellation request ended the workload. - Cancelled, - /// The guest agent could not launch the workload. - LaunchFailed, - /// The workload could not enter its working directory, so nothing of it ran. The guest agent - /// wrote a diagnostic that names the directory to its standard error. - WorkingDirectory, -} - -/// A workload that [`Instance::exec`] started. -pub struct Execution { - /// Exclusive control session that carries this workload's traffic. - session: ControlSession, - /// Identifier correlating requests, output, cancellation, and the final outcome. - request_id: u64, - /// Workload response deadline, absent for a workload without a timeout. - deadline: Option, - /// Maximum time allowed for control traffic after cancellation or failure. - control_timeout: Duration, -} - -impl Execution { - /// Relays the workload's output to `stdout` and `stderr` and returns its outcome. - /// - /// Events are read in short slices, so that setting `cancel` reaches the guest within 20 ms. - /// The guest then terminates the workload and reports the cancelled outcome as usual. Once - /// cancellation is sent, a missing response fails within the control timeout of - /// [`Instance::exec`], or the earlier workload-response deadline, even when the workload has - /// no timeout. The guest agent limits the combined output to [`MAX_OUTPUT_BYTES`], and a - /// guest that sends more fails the execution. If the process that pumps an execution dies, - /// the guest kills the workload as soon as it notices that the control session is gone. - pub fn pump( - mut self, - stdout: &mut dyn OutputSink, - stderr: &mut dyn OutputSink, - cancel: &AtomicBool, - ) -> Result { - let mut deadline = self.deadline; - let mut forwarded = 0usize; - let mut stdout_open = true; - let mut stderr_open = true; - let mut cancel_sent = false; - loop { - if !cancel_sent && cancel.load(Ordering::Acquire) { - let cancellation_deadline = Instant::now() + self.control_timeout; - self.session - .cancel_exec(self.request_id, cancellation_deadline) - .map_err(|error| { - Error::new( - session_code(&error), - format!("cannot deliver the cancellation request: {error}"), - ) - .with_source(error) - })?; - deadline = Some(deadline.map_or(cancellation_deadline, |deadline| { - deadline.min(cancellation_deadline) - })); - cancel_sent = true; - } - let slice = Instant::now() + CANCEL_POLL_INTERVAL; - let poll_deadline = if cancel_sent { - deadline - } else { - Some(deadline.map_or(slice, |deadline| deadline.min(slice))) - }; - let event = match self.session.next_exec_event(self.request_id, poll_deadline) { - Err(SessionError::TimedOut) - if poll_deadline != deadline - && deadline.is_none_or(|deadline| Instant::now() < deadline) => - { - continue; - } - event => event, - }; - let event = event.map_err(|error| match error { - SessionError::TimedOut => { - Error::timed_out("timed out waiting for the workload outcome") - } - ended @ (SessionError::Closed | SessionError::Reset) => Error::new( - session_code(&ended), - "the control session ended before the workload finished; the sandbox VM may \ - have stopped", - ), - other => Error::new( - session_code(&other), - format!("the control session failed: {other}"), - ) - .with_source(other), - })?; - let (chunk, sink, open): (Vec, &mut dyn OutputSink, &mut bool) = match event { - ExecEvent::Stdout(chunk) => (chunk, &mut *stdout, &mut stdout_open), - ExecEvent::Stderr(chunk) => (chunk, &mut *stderr, &mut stderr_open), - ExecEvent::Exit { category, status } => { - return Ok(match category { - ExitCategory::Exit => WorkloadOutcome::Exited(status), - ExitCategory::Signal => WorkloadOutcome::Signaled(if status > 128 { - status - 128 - } else { - status - }), - ExitCategory::Timeout => WorkloadOutcome::TimedOut, - ExitCategory::OutputLimit => WorkloadOutcome::OutputLimitExceeded, - ExitCategory::Failed => WorkloadOutcome::Failed, - ExitCategory::Cancelled => WorkloadOutcome::Cancelled, - }); - } - ExecEvent::Rejected { status, category } => { - return match category.as_str() { - LAUNCH_FAILED => Ok(WorkloadOutcome::LaunchFailed), - // The guest already wrote a diagnostic that names the directory to stderr. - CWD_FAILED => Ok(WorkloadOutcome::WorkingDirectory), - // Only a start that was interrupted leaves a guest without its mappings. - MAPPINGS_INCOMPLETE => Err(Error::new( - ErrorCode::ExecRefused, - "the guest has not mounted every mapped host path, because its start \ - did not complete; stop the sandbox and start it again", - )), - _ => Err(Error::new( - ErrorCode::ExecRefused, - format!( - "the guest agent rejected the workload: {category} (status \ - {status})" - ), - )), - }; - } - }; - forwarded = forwarded.saturating_add(chunk.len()); - if forwarded > MAX_OUTPUT_BYTES { - return Err(Error::protocol( - "the guest exceeded the output limit of the control protocol", - )); - } - if *open && sink.write(&chunk).is_err() { - *open = false; - } - } - } -} - -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn interrupted_writes_leave_temporary_files_that_only_their_removal_takes() { - let directory = tempfile::tempdir().unwrap(); - let names = || { - let mut names: Vec = fs::read_dir(directory.path()) - .unwrap() - .map(|entry| entry.unwrap().file_name().into_string().unwrap()) - .collect(); - names.sort(); - names - }; - let path = directory.path().join("config.json"); - write_json(&path, &serde_json::json!({ "written": true })).unwrap(); - assert_eq!(names(), ["config.json"]); - // Replacements of the file that were interrupted left their temporary files, which other - // files only resemble. - let left = [0, 12345, u64::MAX].map(|number| temporary_name("config.json", number)); - let others = [ - ".config.json.tmp", - ".config.json..tmp", - ".config.json.012.tmp", - ".config.json.+12.tmp", - ".config.json.1a.tmp", - ".config.json.18446744073709551616.tmp", - ".config.json.1.tmp.tmp", - ".xconfig.json.1.tmp", - ".other.json.1.tmp", - "config.json.1.tmp", - ".tmpAbC123", - ]; - for name in left.iter().map(String::as_str).chain(others) { - fs::write(directory.path().join(name), b"{").unwrap(); - } - remove_interrupted_writes(&path).unwrap(); - let mut kept: Vec<&str> = others.into_iter().chain(["config.json"]).collect(); - kept.sort_unstable(); - assert_eq!(names(), kept); - // A directory that does not exist has none. - remove_interrupted_writes(&directory.path().join("gone").join("config.json")).unwrap(); - } - - #[test] - fn readiness_failures_explain_images_without_workload_accounts() { - let hint = "predates workload account creation"; - for ended in [SessionError::ProcessExited, SessionError::Closed] { - assert!(readiness_failure(&ended, true).contains(hint)); - // Only a sandbox that needs the account suggests it. - assert!(!readiness_failure(&ended, false).contains(hint)); - } - for other in [ - SessionError::TimedOut, - SessionError::Protocol("invalid record".to_owned()), - ] { - assert_eq!(readiness_failure(&other, true), other.to_string()); - } - } - - #[test] - fn session_failures_have_the_codes_of_their_kinds_but_a_closed_endpoint() { - assert_eq!( - session_code(&SessionError::Closed), - ErrorCode::EndpointClosed - ); - assert_eq!( - ErrorCode::EndpointClosed.kind(), - session_kind(&SessionError::Closed) - ); - for (error, code) in [ - (SessionError::TimedOut, ErrorCode::TimedOut), - (SessionError::ProcessExited, ErrorCode::Exited), - (SessionError::Reset, ErrorCode::Protocol), - (SessionError::AuthenticationFailed, ErrorCode::Protocol), - ( - SessionError::Protocol("bad".to_owned()), - ErrorCode::Protocol, - ), - (SessionError::Io(io::Error::other("lost")), ErrorCode::Io), - ] { - assert_eq!(session_code(&error), code, "{error}"); - } - } - - /// The instance's store, which exists on Linux and Windows hosts. - #[cfg(any(target_os = "linux", windows))] - mod store { - use serde_json::json; - - use super::super::*; - use crate::host::launch::PROCESS_COMMAND_LINE_LIMIT; - use crate::host::store::{ - ArtifactIdentity, JOURNAL_NAME, LOCK_NAME, LOG_NAME, RUNTIME_NAME, - }; - - /// The configuration record of these tests' caller. - const CONFIG: &str = "config.json"; - - fn instance() -> (tempfile::TempDir, Instance) { - let directory = tempfile::tempdir().unwrap(); - let instance = Instance::create( - directory.path().join("vm"), - "test", - CallerFiles::new(CONFIG).unwrap(), - ) - .unwrap(); - (directory, instance) - } - - /// Returns the runtime identity of the launches of these tests. - fn runtime() -> RuntimeIdentity { - let artifact = |digit: &str| ArtifactIdentity::sha256(&digit.repeat(64)).unwrap(); - RuntimeIdentity::new(artifact("1"), artifact("2"), artifact("3")) - } - - #[test] - fn instances_that_record_no_vm_need_no_runtime_identity() { - let (_directory, instance) = instance(); - let mut locked = instance.lock(None).unwrap(); - let found = locked - .find(|| panic!("an instance that records no VM needs no runtime identity")) - .unwrap(); - assert!(matches!(found, Found::Stopped), "{found:?}"); - drop(locked); - // The instance's caller recorded no configuration, so its lock file went too. - assert!(!instance.dir().join(LOCK_NAME).exists()); - let reopened = - Instance::open(instance.dir(), instance.name(), instance.files().clone()).unwrap(); - assert_eq!(reopened, instance); - } - - #[test] - fn stopped_instances_are_removed_with_the_callers_files() { - let (_directory, instance) = instance(); - for name in [CONFIG, LOG_NAME, RUNTIME_NAME] { - fs::write(instance.dir().join(name), b"{}").unwrap(); - } - // An instance whose store records a VM is not removed. - let locked = instance.lock(None).unwrap(); - assert!(matches!(locked.remove(None), Err(RemovalError::Launched))); - assert!(instance.dir().join(CONFIG).is_file()); - fs::remove_file(instance.dir().join(RUNTIME_NAME)).unwrap(); - // Nor is one whose caller declared no configuration record. - let undeclared = - Instance::open(instance.dir(), instance.name(), CallerFiles::none()).unwrap(); - let locked = undeclared.lock(None).unwrap(); - assert!(matches!(locked.remove(None), Err(RemovalError::Undeclared))); - assert!(instance.dir().join(CONFIG).is_file()); - instance.lock(None).unwrap().remove(None).unwrap(); - assert!(!instance.dir().exists()); - // A transition that comes later finds the instance gone. - let error = instance.lock(None).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::NotFound, "{error}"); - let error = - Instance::open(instance.dir(), instance.name(), CallerFiles::none()).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::NotFound, "{error}"); - } - - #[test] - fn records_of_the_earlier_format_fail_naming_it() { - // Before the store, instances kept runtime records and launch markers of format 3. - for (name, record) in [ - ( - RUNTIME_NAME, - json!({ "format": 3, "pid": 7, "startTime": 8, "endpoint": "e" }), - ), - ( - JOURNAL_NAME, - json!({ "format": 3, "endpoint": "e", "process": { "pid": 7, "startTime": 8 } }), - ), - ] { - let (_directory, instance) = instance(); - let path = instance.dir().join(name); - fs::write(&path, record.to_string()).unwrap(); - let mut locked = instance.lock(None).unwrap(); - let error = locked.find(|| Ok(runtime())).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Io); - assert!( - error.message().contains("has the unsupported format 3"), - "{error}" - ); - assert!( - error - .message() - .contains("stop the sandbox with the build that started it"), - "{error}" - ); - // Nothing is migrated or removed. - assert!(path.is_file()); - } - } - - #[test] - fn launches_are_checked_before_the_instance_exists() { - let directory = tempfile::tempdir().unwrap(); - let mut spec = LaunchSpec::new( - Artifacts::new("openvmm", "vmlinux", "initramfs.cpio.gz"), - crate::Hypervisor::Kvm, - ); - let dir = directory.path().join("vm"); - check_launch(&dir, &spec).unwrap(); - spec.openvmm = directory - .path() - .join("o".repeat(PROCESS_COMMAND_LINE_LIMIT)); - let error = check_launch(&dir, &spec).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported, "{error}"); - assert!(!dir.exists()); - } - } -} diff --git a/nvx-rs/src/host/instance/execution.rs b/nvx-rs/src/host/instance/execution.rs new file mode 100644 index 000000000..8624a5779 --- /dev/null +++ b/nvx-rs/src/host/instance/execution.rs @@ -0,0 +1,181 @@ +//! A running workload, its output, cancellation, and outcome. + +use std::io; +use std::sync::atomic::{AtomicBool, Ordering}; +use std::time::{Duration, Instant}; + +use super::session_code; +use crate::error::{Error, ErrorCode, Result}; +use crate::host::session::{ControlSession, ExecEvent, SessionError}; +use crate::protocol::{ + CWD_FAILED, ExitCategory, LAUNCH_FAILED, MAPPINGS_INCOMPLETE, MAX_OUTPUT_BYTES, +}; + +/// Longest delay between a cancellation request and its delivery to the guest. +const CANCEL_POLL_INTERVAL: Duration = Duration::from_millis(20); + +/// Receives a workload's output. +pub trait OutputSink { + /// Writes one chunk. After an error, the chunks of this stream are discarded, and the + /// workload continues. + fn write(&mut self, chunk: &[u8]) -> io::Result<()>; +} + +impl io::Result<()>> OutputSink for F { + fn write(&mut self, chunk: &[u8]) -> io::Result<()> { + self(chunk) + } +} + +/// How a workload ended. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum WorkloadOutcome { + /// The workload exited with this code. + Exited(i32), + /// A signal with this number ended the workload. + Signaled(i32), + /// The workload's timeout elapsed, and the guest agent killed it. + TimedOut, + /// The workload exceeded [`MAX_OUTPUT_BYTES`], and the guest agent killed it. + OutputLimitExceeded, + /// The guest agent could not supervise the workload. + Failed, + /// A cancellation request ended the workload. + Cancelled, + /// The guest agent could not launch the workload. + LaunchFailed, + /// The workload could not enter its working directory, so nothing of it ran. The guest agent + /// wrote a diagnostic that names the directory to its standard error. + WorkingDirectory, +} + +/// A workload that [`crate::host::instance::Instance::exec`] started. +pub struct Execution { + /// Exclusive control session that carries this workload's traffic. + pub(super) session: ControlSession, + /// Identifier correlating requests, output, cancellation, and the final outcome. + pub(super) request_id: u64, + /// Workload response deadline, absent for a workload without a timeout. + pub(super) deadline: Option, + /// Maximum time allowed for control traffic after cancellation or failure. + pub(super) control_timeout: Duration, +} + +impl Execution { + /// Relays the workload's output to `stdout` and `stderr` and returns its outcome. + /// + /// Events are read in short slices, so that setting `cancel` reaches the guest within 20 ms. + /// The guest then terminates the workload and reports the cancelled outcome as usual. Once + /// cancellation is sent, a missing response fails within the control timeout of + /// [`crate::host::instance::Instance::exec`], or the earlier workload-response deadline, even when the workload has + /// no timeout. The guest agent limits the combined output to [`MAX_OUTPUT_BYTES`], and a + /// guest that sends more fails the execution. If the process that pumps an execution dies, + /// the guest kills the workload as soon as it notices that the control session is gone. + pub fn pump( + mut self, + stdout: &mut dyn OutputSink, + stderr: &mut dyn OutputSink, + cancel: &AtomicBool, + ) -> Result { + let mut deadline = self.deadline; + let mut forwarded = 0usize; + let mut stdout_open = true; + let mut stderr_open = true; + let mut cancel_sent = false; + loop { + if !cancel_sent && cancel.load(Ordering::Acquire) { + let cancellation_deadline = Instant::now() + self.control_timeout; + self.session + .cancel_exec(self.request_id, cancellation_deadline) + .map_err(|error| { + Error::new( + session_code(&error), + format!("cannot deliver the cancellation request: {error}"), + ) + .with_source(error) + })?; + deadline = Some(deadline.map_or(cancellation_deadline, |deadline| { + deadline.min(cancellation_deadline) + })); + cancel_sent = true; + } + let slice = Instant::now() + CANCEL_POLL_INTERVAL; + let poll_deadline = if cancel_sent { + deadline + } else { + Some(deadline.map_or(slice, |deadline| deadline.min(slice))) + }; + let event = match self.session.next_exec_event(self.request_id, poll_deadline) { + Err(SessionError::TimedOut) + if poll_deadline != deadline + && deadline.is_none_or(|deadline| Instant::now() < deadline) => + { + continue; + } + event => event, + }; + let event = event.map_err(|error| match error { + SessionError::TimedOut => { + Error::timed_out("timed out waiting for the workload outcome") + } + ended @ (SessionError::Closed | SessionError::Reset) => Error::new( + session_code(&ended), + "the control session ended before the workload finished; the sandbox VM may \ + have stopped", + ), + other => Error::new( + session_code(&other), + format!("the control session failed: {other}"), + ) + .with_source(other), + })?; + let (chunk, sink, open): (Vec, &mut dyn OutputSink, &mut bool) = match event { + ExecEvent::Stdout(chunk) => (chunk, &mut *stdout, &mut stdout_open), + ExecEvent::Stderr(chunk) => (chunk, &mut *stderr, &mut stderr_open), + ExecEvent::Exit { category, status } => { + return Ok(match category { + ExitCategory::Exit => WorkloadOutcome::Exited(status), + ExitCategory::Signal => WorkloadOutcome::Signaled(if status > 128 { + status - 128 + } else { + status + }), + ExitCategory::Timeout => WorkloadOutcome::TimedOut, + ExitCategory::OutputLimit => WorkloadOutcome::OutputLimitExceeded, + ExitCategory::Failed => WorkloadOutcome::Failed, + ExitCategory::Cancelled => WorkloadOutcome::Cancelled, + }); + } + ExecEvent::Rejected { status, category } => { + return match category.as_str() { + LAUNCH_FAILED => Ok(WorkloadOutcome::LaunchFailed), + // The guest already wrote a diagnostic that names the directory to stderr. + CWD_FAILED => Ok(WorkloadOutcome::WorkingDirectory), + // Only a start that was interrupted leaves a guest without its mappings. + MAPPINGS_INCOMPLETE => Err(Error::new( + ErrorCode::ExecRefused, + "the guest has not mounted every mapped host path, because its start \ + did not complete; stop the sandbox and start it again", + )), + _ => Err(Error::new( + ErrorCode::ExecRefused, + format!( + "the guest agent rejected the workload: {category} (status \ + {status})" + ), + )), + }; + } + }; + forwarded = forwarded.saturating_add(chunk.len()); + if forwarded > MAX_OUTPUT_BYTES { + return Err(Error::protocol( + "the guest exceeded the output limit of the control protocol", + )); + } + if *open && sink.write(&chunk).is_err() { + *open = false; + } + } + } +} diff --git a/nvx-rs/src/host/instance/files.rs b/nvx-rs/src/host/instance/files.rs new file mode 100644 index 000000000..000ca7128 --- /dev/null +++ b/nvx-rs/src/host/instance/files.rs @@ -0,0 +1,133 @@ +//! Private files of an instance's caller and the temporary files of their replacements. + +use std::fs::{self, OpenOptions}; +use std::io::{self, Write}; +use std::path::Path; + +use serde::Serialize; +use serde::de::DeserializeOwned; + +use crate::error::{Error, Result}; + +/// Builds an error mapper that preserves an I/O source under `context`. +pub(super) fn io_error(context: String) -> impl FnOnce(io::Error) -> Error { + move |error| Error::io(context).with_source(error) +} + +/// Returns create-and-write options that request owner-only permissions on Unix. +fn private_options() -> OpenOptions { + let mut options = OpenOptions::new(); + options.write(true).create(true); + #[cfg(unix)] + std::os::unix::fs::OpenOptionsExt::mode(&mut options, 0o600); + options +} + +/// Returns the name of the temporary file through which [`write_private`] replaces the file `name` +/// for the random `number`: `.NAME.NUMBER.tmp`, which stays beside it if the replacement is +/// interrupted. +pub(super) fn temporary_name(name: &str, number: u64) -> String { + format!(".{name}.{number}.tmp") +} + +/// Atomically replaces `path` with `contents`, readable only by the current user, as callers do +/// with the configuration records that they keep beside an instance's store. +/// +/// The contents go to a temporary file beside `path` first, which a replacement that is +/// interrupted leaves, and which [`remove_interrupted_writes`] removes. +pub fn write_private(path: &Path, contents: &[u8]) -> Result<()> { + let mut suffix = [0u8; 8]; + getrandom::fill(&mut suffix) + .map_err(|error| Error::other("cannot name a temporary file").with_source(error))?; + let name = path + .file_name() + .map(|name| name.to_string_lossy().into_owned()) + .unwrap_or_default(); + let temporary = path.with_file_name(temporary_name(&name, u64::from_le_bytes(suffix))); + let written = (|| -> io::Result<()> { + let mut file = private_options().create_new(true).open(&temporary)?; + file.write_all(contents)?; + file.sync_all()?; + drop(file); + fs::rename(&temporary, path) + })(); + if written.is_err() { + let _ = fs::remove_file(&temporary); + } + written.map_err(io_error(format!("cannot write {}", path.display()))) +} + +/// Atomically replaces `path` with `value` as pretty JSON and a newline, readable only by the +/// current user. +pub fn write_json(path: &Path, value: &T) -> Result<()> { + let mut text = serde_json::to_vec_pretty(value) + .map_err(|error| Error::other("cannot encode sandbox state").with_source(error))?; + text.push(b'\n'); + write_private(path, &text) +} + +/// Reads the JSON at `path`, or returns `None` if the file does not exist. +pub fn read_json(path: &Path) -> Result> { + let text = match fs::read(path) { + Ok(text) => text, + Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(None), + Err(error) => return Err(io_error(format!("cannot read {}", path.display()))(error)), + }; + serde_json::from_slice(&text).map(Some).map_err(|error| { + Error::io(format!("sandbox state {} is malformed", path.display())).with_source(error) + }) +} + +/// Removes the file at `path` if it exists. +pub fn remove_if_present(path: &Path) -> Result<()> { + match fs::remove_file(path) { + Err(error) if error.kind() != io::ErrorKind::NotFound => { + Err(io_error(format!("cannot remove {}", path.display()))(error)) + } + _ => Ok(()), + } +} + +/// Removes the temporary files that interrupted replacements of `path` with [`write_private`] or +/// [`write_json`] left beside it, and keeps every other file. A directory that does not exist has +/// none. +/// +/// The store's removal refuses those files, which are neither the store's nor the files that its +/// caller declares, so a caller that writes its files with [`write_private`] removes them first, +/// under the instance's lock. +pub fn remove_interrupted_writes(path: &Path) -> Result<()> { + let Some(name) = path.file_name().map(|name| name.to_string_lossy()) else { + return Ok(()); + }; + let dir = match path.parent() { + Some(dir) if !dir.as_os_str().is_empty() => dir, + _ => Path::new("."), + }; + let unlisted = || io_error(format!("cannot list {}", dir.display())); + let entries = match fs::read_dir(dir) { + Ok(entries) => entries, + Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(()), + Err(error) => return Err(unlisted()(error)), + }; + for entry in entries { + let entry = entry.map_err(unlisted())?; + if entry + .file_name() + .to_str() + .is_some_and(|entry| is_temporary_of(entry, &name)) + { + remove_if_present(&entry.path())?; + } + } + Ok(()) +} + +/// Returns whether `entry` is the name of a temporary file through which [`write_private`] +/// replaces the file `name`. +fn is_temporary_of(entry: &str, name: &str) -> bool { + entry + .strip_suffix(".tmp") + .and_then(|rest| rest.rsplit_once('.')) + .and_then(|(_, number)| number.parse::().ok()) + .is_some_and(|number| temporary_name(name, number) == entry) +} diff --git a/nvx-rs/src/host/instance/locked.rs b/nvx-rs/src/host/instance/locked.rs new file mode 100644 index 000000000..2f5b418c6 --- /dev/null +++ b/nvx-rs/src/host/instance/locked.rs @@ -0,0 +1,286 @@ +//! The lifecycle transitions that hold an instance's lock. + +use std::time::{Duration, Instant}; + +use super::{ + Found, Instance, Mismatched, Running, Started, Stopped, control_endpoint, generate_capability, + readiness_failure, session_code, +}; +use crate::error::{Error, ErrorCode, Result}; +use crate::host::launch::{LaunchSpec, timeout_problem}; +use crate::host::process::KILL_TIMEOUT; +use crate::host::report; +use crate::host::session::{self, ControlSession, SessionError}; +use crate::host::store::{LaunchId, LockedStore, RemovalError, RuntimeIdentity}; +use crate::host::supervisor::{self, Reattached, StopOutcome, VmmCommand}; +use crate::protocol::GuestFeatures; + +/// An instance whose lock this holds, through which its lifecycle transitions change its +/// records. Dropping it releases the lock. +#[derive(Debug)] +pub struct LockedInstance { + /// The instance. + pub(super) instance: Instance, + /// Its locked store. + pub(super) locked: LockedStore, +} + +impl LockedInstance { + /// Returns the instance. + pub fn instance(&self) -> &Instance { + &self.instance + } + + /// Finds the instance's VM with [`supervisor::reattach`], under the runtime identity that + /// `runtime` returns: the one with which the caller would launch it now. + /// + /// `runtime` is asked only when the store records a VM or a start, so that an instance that + /// never started, or that stopped, needs no runtime identity. Reattach ends a start that was + /// interrupted, signaling only an OpenVMM process that it verified, and forgets a VM that + /// exited, as one that a record of an earlier host boot names has. Fails as `runtime` fails, + /// for records that the store does not read, such as those of an earlier format, whose + /// message names it, and when whether the VM runs cannot be determined, because an + /// interrupted start cannot be ended yet or the recorded process cannot be examined: the + /// records then remain for a later attempt. + /// + /// A VM whose capability cannot be read, as when its file is missing or damaged, still runs, + /// and is found [`Found::Running`]: [`Instance::exec`] then fails as reading it failed, and + /// [`stop`](Self::stop) ends the VM through its verified process identity. + pub fn find(&mut self, runtime: impl FnOnce() -> Result) -> Result { + if self.locked.journal()?.is_none() && self.locked.runtime()?.is_none() { + return Ok(Found::Stopped); + } + let runtime = runtime()?; + Ok(match supervisor::reattach(&mut self.locked, &runtime)? { + Reattached::Live(vmm) => Found::Running(Running { + vmm, + capability: self.locked.read_capability(), + }), + Reattached::Mismatched { vmm, differences } => Found::Mismatched(Mismatched { + name: self.instance.name.clone(), + vmm, + differences, + }), + Reattached::Exited { .. } => Found::Stopped, + Reattached::Indeterminate(reason) => { + return Err(Error::new( + ErrorCode::VmmIndeterminate, + format!( + "cannot determine whether the OpenVMM process of sandbox {} runs: \ + {reason}", + self.instance.name + ), + )); + } + }) + } + + /// Launches the VM that `spec` describes under the runtime identity `runtime`, and waits up + /// to `timeout` until its guest is ready. + /// + /// The caller has found the instance stopped with [`find`](Self::find). Start first checks the + /// export plan with [`ExportPlan::verify`](crate::host::filesystem::ExportPlan::verify): that the + /// host keeps its read-only paths read-only, and that the mapped host paths still name the + /// objects they named when they were planned. It launches OpenVMM with + /// [`supervisor::launch`], whose journal records the launch before OpenVMM is spawned and its + /// process identity right after, so that [`find`](Self::find) ends a start whose caller died. + /// The capability travels through OpenVMM's standard-input pipe, which is closed before + /// OpenVMM starts, and never appears in arguments, the environment, or the log. Once the guest + /// is ready, has the required features, and has mounted the mapped host paths, start commits + /// the launch: the runtime record names the VM from then on. + /// + /// Fails with [`crate::ErrorKind::Unavailable`], and the code [`ErrorCode::GuestFeaturesMissing`], + /// when the guest image lacks a feature in [`GuestFeatures::REQUIRED`], and with + /// [`crate::ErrorKind::Unsupported`] when `timeout` is zero or + /// longer than [`MAX_TIMEOUT`](crate::host::launch::MAX_TIMEOUT), when the export plan maps a + /// read-only path that the host cannot keep read-only, when the specification fails + /// [`LaunchSpec::check`] or exceeds a limit, or when OpenVMM's command line would not fit + /// [`PROCESS_COMMAND_LINE_LIMIT`](crate::host::launch::PROCESS_COMMAND_LINE_LIMIT), which holds on + /// every host; such a launch records nothing. Unlike [`crate::host::instance::check_launch`], start keeps no reserve. + /// A launch that does not become ready is terminated through its verified process identity; + /// one that cannot be ended keeps its journal, and the next transition ends it. + pub fn start( + &mut self, + spec: &LaunchSpec, + runtime: &RuntimeIdentity, + timeout: Duration, + ) -> Result { + check_timeout("start timeout", timeout, true)?; + if let Some(plan) = &spec.filesystem { + plan.verify()?; + } + let instance = &self.instance; + let report = instance.report_path(); + // Every control endpoint of the instance has the same length, so a launch that would not + // fit is refused before the store records it. + let endpoint = control_endpoint(&instance.store.socket_path())?; + spec.fit_process_command_line(&spec.openvmm_arguments(&endpoint, &report)?, 0)?; + let capability = generate_capability()?; + let launch = LaunchId::generate()?; + let started = Instant::now(); + let starting = supervisor::launch( + &mut self.locked, + &launch, + runtime, + &capability, + |endpoint| { + let arguments = spec.openvmm_arguments(endpoint, &report)?; + spec.fit_process_command_line(&arguments, 0)?; + let mut command = VmmCommand::new(&spec.openvmm, arguments, instance.dir()); + command.breakaway_from_job = spec.breakaway_from_job; + Ok(command) + }, + )?; + let (pid, start_time) = { + let reference = starting.process().reference(); + (reference.pid, reference.start_time) + }; + let deadline = started + timeout; + let ready = session::connect(starting.endpoint(), pid, start_time, deadline) + .and_then(|transport| ControlSession::attach(transport, &capability, deadline)) + .and_then(|mut session| { + session.ping(deadline)?; + let features = session.features(deadline)?; + Ok((session, features)) + }); + let (mut session, features) = match ready { + Ok(ready) => ready, + Err(error) => { + let reason = readiness_failure(&error, spec.identity.create_account); + return Err(instance.abort_start(starting, session_code(&error), &reason)); + } + }; + let missing = features.missing(GuestFeatures::REQUIRED); + if !missing.is_empty() { + drop(session); + return Err(instance.reject_guest(starting, &missing)); + } + // Workloads run only once the guest has mounted every mapped host path. + if let Some(plan) = &spec.filesystem + && let Err(error) = session.map_host_paths(&plan.guest_table(), deadline) + { + drop(session); + let reason = format!("the host paths were not mapped: {error}"); + return Err(instance.abort_start(starting, session_code(&error), &reason)); + } + drop(session); + let vmm = starting.commit()?; + Ok(Started { + runtime: vmm.record().clone(), + boot: started.elapsed(), + }) + } + + /// Asks the guest of the running VM `running` to shut down, and terminates OpenVMM through + /// its verified process identity if it has not exited within `timeout`, as [`crate::host::supervisor::Vmm::stop`] + /// does with a forced step, or at once when the guest cannot be asked, as without the VM's + /// capability. + /// + /// A running execution keeps the control session until it ends, so a stop that cannot reach + /// the guest in time terminates the VM, and the execution then fails. Once OpenVMM has + /// exited, the stop clears the runtime record and reads OpenVMM's outcome report. Fails with + /// [`crate::ErrorKind::Unsupported`] when `timeout` is zero or longer than + /// [`MAX_TIMEOUT`](crate::host::launch::MAX_TIMEOUT), and when OpenVMM cannot be terminated, or does + /// not exit after it was, which keeps its runtime record. + pub fn stop(&mut self, running: Running, timeout: Duration) -> Result { + check_timeout("stop timeout", timeout, true)?; + let deadline = Instant::now() + timeout; + let Running { vmm, capability } = running; + let stopped = vmm.stop( + &mut self.locked, + deadline, + Some(KILL_TIMEOUT), + |vmm, deadline| { + // Without its capability, no session can reach the guest, and the forced step + // ends the VM. + let capability = + capability.map_err(|error| SessionError::Protocol(error.to_string()))?; + let process = vmm.record().process(); + let transport = + session::connect(vmm.endpoint(), process.pid, process.start_time, deadline)?; + ControlSession::attach(transport, &capability, deadline)?.stop(deadline) + }, + )?; + let forced = match stopped.outcome { + StopOutcome::Graceful => false, + StopOutcome::Forced => true, + StopOutcome::TimedOut => { + return Err(Error::other(format!( + "OpenVMM process {} of sandbox {} did not exit after it was terminated", + vmm.record().process().pid, + self.instance.name + ))); + } + }; + Ok(Stopped { + forced, + report: report::read(&self.instance.report_path()), + }) + } + + /// Stops the VM `mismatched`, which runs under another runtime identity than the caller's, by + /// identity-verified forced termination: terminates OpenVMM through its verified process + /// identity, as [`crate::host::supervisor::Vmm::terminate`] does, and waits up to `timeout` for it to exit. + /// + /// The VM's guest might not speak this build's control protocol, so the stop opens no control + /// session to it, and does not ask it to shut down: only the OpenVMM process that + /// [`find`](Self::find) verified is signaled. Once OpenVMM has exited, the stop clears the + /// runtime record. It reads no outcome report, so [`Stopped::report`] is `None`: a terminated + /// OpenVMM writes none, and one that exited before it was terminated wrote it under the other + /// runtime's contracts. [`Stopped::forced`] is false only for an OpenVMM that exited before it + /// was terminated. + /// + /// Fails with [`crate::ErrorKind::Unsupported`] when `timeout` is zero or longer than + /// [`MAX_TIMEOUT`](crate::host::launch::MAX_TIMEOUT), and when OpenVMM's state cannot be + /// determined, OpenVMM cannot be terminated, or it does not exit within `timeout`: its runtime + /// record then remains, and a later stop retries. + pub fn terminate(&mut self, mismatched: Mismatched, timeout: Duration) -> Result { + check_timeout("stop timeout", timeout, true)?; + let Mismatched { vmm, .. } = mismatched; + let forced = match vmm.terminate(&mut self.locked, timeout)? { + StopOutcome::Graceful => false, + StopOutcome::Forced => true, + StopOutcome::TimedOut => { + return Err(Error::other(format!( + "OpenVMM process {} of sandbox {} did not exit after it was terminated", + vmm.record().process().pid, + self.instance.name + ))); + } + }; + Ok(Stopped { + forced, + report: None, + }) + } + + /// Removes the stopped instance: its store's records and files, the caller's files that it + /// declared, the configuration record last, and its directory, through the store's + /// [`LockedStore::remove`], and releases the lock. Open files, such as the lock files of other + /// transitions, that keep the emptied directory are then waited for until `deadline`, or for + /// as long as it takes without one, as [`Lingering::wait`](crate::host::store::Lingering::wait) + /// does. + /// + /// The caller has found the instance stopped with [`find`](Self::find): the removal refuses, + /// and removes nothing, while the store records a VM or a start, and for an instance whose + /// caller declared no configuration record. Entries in the directory that are neither the + /// store's nor the caller's are refused, and kept with the directory, once the rest is gone, + /// among them the temporary files that interrupted writes of the caller's files left, which + /// the caller removes first with [`crate::host::instance::remove_interrupted_writes`]. + pub fn remove(self, deadline: Option) -> std::result::Result<(), RemovalError> { + match self.locked.remove()? { + None => Ok(()), + Some(mut lingering) => lingering.wait(deadline), + } + } +} + +/// Fails with [`crate::ErrorKind::Unsupported`] when the duration that `name` describes is longer than +/// [`MAX_TIMEOUT`](crate::host::launch::MAX_TIMEOUT), or zero while it must be `positive`, so that no +/// deadline derived from it overflows. +pub(super) fn check_timeout(name: &str, value: Duration, positive: bool) -> Result<()> { + match timeout_problem(value, positive) { + Some(problem) => Err(Error::unsupported(format!("the {name} {problem}"))), + None => Ok(()), + } +} diff --git a/nvx-rs/src/host/instance/mod.rs b/nvx-rs/src/host/instance/mod.rs new file mode 100644 index 000000000..b604d2b52 --- /dev/null +++ b/nvx-rs/src/host/instance/mod.rs @@ -0,0 +1,485 @@ +//! Supervision of one VM instance: its OpenVMM process, its control endpoint, and the records +//! that name them, through the [`store`](crate::host::store), the [`crate::host::supervisor`], and the control +//! [`session`]. +//! +//! An instance lives in a [`SandboxStore`] of its own: a directory private to the current user, +//! in which the store keeps the instance's lock, the journal of its launch, and the runtime +//! record of its VM, and in which the caller may keep files of its own under other names, which it +//! declares as [`CallerFiles`], such as the configuration record whose presence means that the +//! instance is provisioned: +//! +//! ```text +//! / +//! sandbox.lock the lock that serializes the instance's lifecycle transitions +//! launch.json the journal of a start in progress, or of one that was interrupted +//! runtime.json the runtime record of the running VM +//! control.capability launch capability (running only) +//! control.sock Linux control endpoint (running only) +//! openvmm.log OpenVMM output of the latest start +//! outcome.json OpenVMM outcome report of the latest run +//! ... the caller's files +//! ``` +//! +//! Lifecycle transitions run under the instance's lock, through the [`LockedInstance`] that +//! [`Instance::lock`] returns, so that every operation can run in a different process. The lock is +//! the store's [lifecycle lock](crate::host::store#lifecycle-locks) of an existing sandbox, which knows +//! the caller's files: +//! +//! - [`LockedInstance::find`] finds the instance's VM as an orchestrator does after it restarts +//! within the same host boot, with [`crate::host::supervisor::reattach`]: it ends a start whose caller died, +//! forgets a VM that exited, and compares the caller's runtime identity with the one that a +//! running VM was launched with. A VM of another runtime identity is [`Found::Mismatched`]: it +//! runs through artifacts or contracts that the caller does not have, so the instance does not +//! drive it, and only ends it. +//! - [`LockedInstance::start`] launches a detached OpenVMM process with the managed lifecycle +//! through [`crate::host::supervisor::launch`], passes it a fresh 32-byte capability through standard input, +//! and waits until the guest agent answers on the authenticated control console. The agent must +//! also advertise the control features in [`crate::protocol::GuestFeatures::REQUIRED`]; a guest image that lacks +//! one is terminated, because it would silently ignore the policy or request that needs the +//! feature. Start then hands the agent the table of mapped host paths, and fails if the agent +//! cannot mount them all. +//! - [`LockedInstance::stop`] asks the guest to shut down and falls back to terminating OpenVMM +//! through its verified process identity. +//! - [`LockedInstance::terminate`] stops a mismatched VM: it terminates OpenVMM through its +//! verified process identity, opens no control session, whose protocol the guest might not +//! speak, and reads no outcome report, which follows the other runtime's contracts. +//! - [`LockedInstance::remove`] removes a stopped instance, with the caller's files, and its +//! directory, through the store's [removal](crate::host::store#removal). +//! +//! [`Instance::exec`] runs a workload through the control console of a [`Running`] VM without the +//! lock, and [`Execution::pump`] streams its output live. The guest serializes executions: a +//! second execution waits until the first finishes. Messages call the instance `sandbox NAME`, +//! after the name that [`Instance::create`] or [`Instance::open`] gives it. + +mod execution; +mod files; +mod locked; + +use std::fmt; +use std::io; +use std::path::{Path, PathBuf}; +use std::time::{Duration, Instant}; + +use crate::error::{Error, ErrorCode, ErrorKind, Result}; +use crate::host::artifacts::Artifacts; +use crate::host::launch::LaunchSpec; +use crate::host::platform; +use crate::host::report::Report; +use crate::host::session::{self, ControlSession, SessionError}; +use crate::host::store::{ + CallerFiles, IdentityPart, OUTCOME_NAME, Recovery, RuntimeRecord, SOCKET_NAME, SandboxStore, +}; +use crate::host::supervisor::{Starting, Vmm}; +use crate::protocol::{CAPABILITY_LEN, Workload}; + +pub use self::execution::{Execution, OutputSink, WorkloadOutcome}; +use self::files::io_error; +pub use self::files::{ + read_json, remove_if_present, remove_interrupted_writes, write_json, write_private, +}; +pub use self::locked::LockedInstance; +use self::locked::check_timeout; + +/// Returns a fresh launch capability, which is never all zero. +pub fn generate_capability() -> Result<[u8; CAPABILITY_LEN]> { + let mut capability = [0u8; CAPABILITY_LEN]; + while capability == [0; CAPABILITY_LEN] { + getrandom::fill(&mut capability).map_err(|error| { + Error::other("cannot generate a control capability").with_source(error) + })?; + } + Ok(capability) +} + +/// Returns a control endpoint of an instance whose Linux control socket is `socket`: the socket +/// on Linux, and a fresh named pipe, whose name always has the same length, on Windows, as a +/// launch chooses it. +fn control_endpoint(socket: &Path) -> Result { + platform::control_endpoint(socket) + .map_err(|error| Error::io("cannot choose a control endpoint").with_source(error)) +} + +/// Fails with [`ErrorKind::Unsupported`] if OpenVMM's command line for `spec`, launched for an +/// instance in the directory `dir`, would not fit the limit of a Windows process command line; +/// see [`LaunchSpec::check_process_command_line`]. +/// +/// The directory need not exist yet, so that a caller can check a launch before it creates the +/// instance. Every control endpoint of an instance has the same length. +pub fn check_launch(dir: &Path, spec: &LaunchSpec) -> Result<()> { + let dir = std::path::absolute(dir).map_err(io_error(format!( + "cannot resolve the instance directory {}", + dir.display() + )))?; + let endpoint = control_endpoint(&dir.join(SOCKET_NAME))?; + spec.check_process_command_line(&endpoint, &dir.join(OUTCOME_NAME)) +} + +/// Classifies a session failure. +fn session_kind(error: &SessionError) -> ErrorKind { + match error { + SessionError::TimedOut => ErrorKind::TimedOut, + SessionError::ProcessExited => ErrorKind::Exited, + SessionError::Refused(_) | SessionError::Io(_) => ErrorKind::Io, + SessionError::Closed + | SessionError::Reset + | SessionError::AuthenticationFailed + | SessionError::Protocol(_) => ErrorKind::Protocol, + } +} + +/// Returns the code of a session failure: [`ErrorCode::EndpointClosed`] for a session that the +/// control endpoint closed, as OpenVMM closes it when the VM shuts down, and otherwise the code of +/// its [kind](session_kind). +fn session_code(error: &SessionError) -> ErrorCode { + match error { + SessionError::Closed => ErrorCode::EndpointClosed, + other => session_kind(other).into(), + } +} + +/// Explains why a launch did not become ready. +fn readiness_failure(error: &SessionError, runs_as_host_user: bool) -> String { + let mut reason = error.to_string(); + // An image that predates account creation ends its boot, before the guest agent starts, when + // it finds no account for the host user that the workloads run as. The VM then vanishes, or + // its control endpoint closes first. + let vm_ended = matches!(error, SessionError::ProcessExited | SessionError::Closed); + if runs_as_host_user && vm_ended { + reason.push_str( + "; a guest image that predates workload account creation exits during boot when \ + workloads run under the host user's IDs, so use an NVX release or build that \ + includes it", + ); + } + reason +} + +/// A VM that [`LockedInstance::start`] launched and the guest agent of which is ready. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Started { + /// The runtime record of its launch: its OpenVMM process, its control endpoint, and the + /// runtime identity that it was launched with. + pub runtime: RuntimeRecord, + /// Time from launch to guest readiness. + pub boot: Duration, +} + +/// A VM that [`LockedInstance::stop`] stopped, or that [`LockedInstance::terminate`] ended. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Stopped { + /// Whether OpenVMM was terminated: because the guest did not shut down in time, or because + /// the VM ran under another runtime identity. + pub forced: bool, + /// Summary of OpenVMM's outcome report, if it wrote one. [`LockedInstance::terminate`] reads + /// none, since the other runtime's OpenVMM wrote it. + pub report: Option, +} + +/// The running VM of an instance, which [`LockedInstance::find`] found under the caller's runtime +/// identity, and the capability with which its control sessions attach. +pub struct Running { + /// The VM, held through its verified process identity. + vmm: Vmm, + /// The launch capability of the VM, or the error with which reading it failed: only the + /// VM's control sessions need it, and a stop ends the VM without one. + capability: Result<[u8; CAPABILITY_LEN]>, +} + +impl Running { + /// Returns the VM, held through its verified process identity. + pub fn vmm(&self) -> &Vmm { + &self.vmm + } + + /// Returns the runtime record of the VM's launch. + pub fn runtime(&self) -> &RuntimeRecord { + self.vmm.record() + } + + /// Returns the launch capability of the VM, or fails as reading it failed, with the same + /// code, description, and the description of its cause. + fn capability(&self) -> Result<&[u8; CAPABILITY_LEN]> { + self.capability.as_ref().map_err(|error| { + let copy = Error::new(error.code(), error.message()); + match std::error::Error::source(error) { + Some(cause) => copy.with_source(io::Error::other(cause.to_string())), + None => copy, + } + }) + } +} + +impl fmt::Debug for Running { + fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { + // The capability authenticates control sessions, so it is never shown. + formatter + .debug_struct("Running") + .field("vmm", &self.vmm) + .finish_non_exhaustive() + } +} + +/// The running VM of an instance, which [`LockedInstance::find`] found under another runtime +/// identity than the caller's, and which [`LockedInstance::terminate`] ends. +#[derive(Debug)] +pub struct Mismatched { + /// The instance's name in messages. + name: String, + /// The VM, held through its verified process identity. + vmm: Vmm, + /// The parts of the runtime identity that differ. + differences: Vec, +} + +impl Mismatched { + /// Returns the VM, held through its verified process identity. + pub fn vmm(&self) -> &Vmm { + &self.vmm + } + + /// Returns the parts of the runtime identity that differ, as + /// [`crate::host::store::RuntimeIdentity::differences`] returns them. + pub fn differences(&self) -> &[IdentityPart] { + &self.differences + } + + /// Returns the error that reports the VM, which the instance does not drive, since only the + /// build and artifacts that started it speak its contracts: it names the parts that differ, + /// and that stopping the VM terminates it. Its code is [`ErrorCode::RuntimeMismatch`]. + pub fn error(&self) -> Error { + let parts: Vec<&str> = self.differences.iter().map(|part| part.name()).collect(); + Error::new( + ErrorCode::RuntimeMismatch, + format!( + "sandbox {} runs under another runtime identity, which differs in {}, so it is \ + not driven: stop it to terminate its OpenVMM process {} without a guest shutdown", + self.name, + parts.join(", "), + self.vmm.record().process().pid + ), + ) + } +} + +/// What [`LockedInstance::find`] found of an instance's VM. +#[derive(Debug)] +pub enum Found { + /// No VM of the instance runs: none was started, the last one stopped or exited, or a start + /// that was interrupted is over. + Stopped, + /// The VM runs under the caller's runtime identity. + Running(Running), + /// The VM runs under another runtime identity: the caller ends it with + /// [`LockedInstance::terminate`], or reports it, as [`Mismatched::error`] does. + Mismatched(Mismatched), +} + +/// One VM instance in a sandbox store of its own. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Instance { + /// The store of the instance's runtime records, logs, and control endpoint. + store: SandboxStore, + /// Sandbox name used in diagnostics. + name: String, + /// The files that the caller keeps in the instance's directory. + files: CallerFiles, +} + +impl Instance { + /// Creates the directory `dir` of a new instance, whose parent must exist, private to the + /// current user, as [`SandboxStore::create`] does, and supervises the instance in it, which + /// messages call `sandbox NAME`, and in which the caller keeps `files`. + pub fn create( + dir: impl Into, + name: impl Into, + files: CallerFiles, + ) -> Result { + Ok(Self { + store: SandboxStore::create(dir)?, + name: name.into(), + files, + }) + } + + /// Supervises the instance in the existing directory `dir`, which must be private to the + /// current user, as [`SandboxStore::open`] requires, which messages call `sandbox NAME`, and + /// in which the caller keeps `files`. Fails with [`ErrorKind::NotFound`] if `dir` does not + /// exist. + pub fn open( + dir: impl Into, + name: impl Into, + files: CallerFiles, + ) -> Result { + Ok(Self { + store: SandboxStore::open(dir)?, + name: name.into(), + files, + }) + } + + /// Returns the store of the instance's runtime records. + pub fn store(&self) -> &SandboxStore { + &self.store + } + + /// Returns the instance's directory, as an absolute path. + pub fn dir(&self) -> &Path { + self.store.dir() + } + + /// Returns the instance's name in messages. + pub fn name(&self) -> &str { + &self.name + } + + /// Returns the files that the caller keeps in the instance's directory. + pub fn files(&self) -> &CallerFiles { + &self.files + } + + /// Returns the path of the OpenVMM log of the latest start. + pub fn log_path(&self) -> PathBuf { + self.store.log_path() + } + + /// Returns the path of OpenVMM's outcome report of the latest run. + pub fn report_path(&self) -> PathBuf { + self.store.outcome_path() + } + + /// Takes the instance's lock, which serializes its lifecycle transitions, waiting for it + /// until `deadline`, or for as long as it takes without one, as + /// [`SandboxStore::lock_existing`] does with the caller's files. + /// + /// Fails with [`ErrorKind::NotFound`] once the instance's directory is gone, as a removal + /// that this waited for leaves it, and with [`ErrorKind::TimedOut`] when the deadline passes + /// first. A caller that keeps a configuration record checks it under the lock: the lock of a + /// directory without it is released without a lock file. + pub fn lock(&self, deadline: Option) -> Result { + Ok(LockedInstance { + locked: SandboxStore::lock_existing(self.dir(), &self.files, deadline)?, + instance: self.clone(), + }) + } + + /// Describes why this instance did not start and points the caller to its log. + fn start_failure(&self, code: impl Into, reason: &str) -> Error { + Error::new( + code, + format!( + "sandbox {} failed to start: {reason}; see {}", + self.name, + self.log_path().display() + ), + ) + } + + /// Ends a launch that did not become ready, as [`Starting::abandon`] does, and describes why + /// the instance did not start. + /// + /// A launch whose OpenVMM cannot be ended keeps its journal, and the next transition of the + /// instance ends it, when [`LockedInstance::find`] recovers it. + fn abort_start( + &self, + starting: Starting<'_>, + code: impl Into, + reason: &str, + ) -> Error { + let code = code.into(); + let unended = match starting.abandon() { + Ok(Recovery::Indeterminate(unended)) => unended, + Ok(_) => return self.start_failure(code, reason), + Err(error) => error.to_string(), + }; + self.start_failure( + code, + &format!( + "{reason}; the launch could not be ended, and remains for recovery: {unended}" + ), + ) + } + + /// Ends a launch whose guest image lacks control features that NVX needs. + /// + /// An older image would boot and answer, but it would silently ignore host path mappings and + /// cancellation requests, so the VM must not run at all. + fn reject_guest(&self, starting: Starting<'_>, missing: &[&str]) -> Error { + let reason = format!( + "the guest image lacks control features that the openvmm backend needs ({}); use an \ + NVX release or build that includes them, for example through {}", + missing.join(", "), + Artifacts::ENV_DIR + ); + self.abort_start(starting, ErrorCode::GuestFeaturesMissing, &reason) + } + + /// Maps a control-session failure onto an instance-facing error and message. + fn session_error(&self, error: SessionError, control_timeout: Duration) -> Error { + match error { + SessionError::ProcessExited => Error::new( + ErrorCode::SandboxExited, + format!("sandbox {} stopped unexpectedly", self.name), + ), + SessionError::TimedOut => Error::timed_out(format!( + "sandbox {} did not accept a control session within {} s; another exec may \ + still be running", + self.name, + control_timeout.as_secs() + )), + other => Error::new( + session_code(&other), + format!( + "the control session with sandbox {} failed: {other}", + self.name + ), + ) + .with_source(other), + } + } + + /// Starts `workload` in the instance's running VM, `running`, without the instance's lock. + /// + /// It waits up to `control_timeout` for the control session, which the guest grants one + /// execution at a time. [`Execution::pump`] then waits for the outcome up to the workload's + /// own timeout plus `response_grace`, and without a limit when the workload has no timeout. + /// Fails with [`ErrorKind::Exited`], and the code [`ErrorCode::SandboxExited`], when the VM is + /// gone, and with [`ErrorKind::Unsupported`] when `control_timeout` is zero or either + /// duration is longer than [`MAX_TIMEOUT`](crate::host::launch::MAX_TIMEOUT). Fails as reading the + /// capability failed when [`find`](LockedInstance::find) could not read it. + pub fn exec( + &self, + running: &Running, + workload: &Workload<'_>, + control_timeout: Duration, + response_grace: Duration, + ) -> Result { + check_timeout("control timeout", control_timeout, true)?; + check_timeout("response grace", response_grace, false)?; + let capability = running.capability()?; + let deadline = Instant::now() + control_timeout; + let process = running.vmm.record().process(); + let mut session = session::connect( + running.vmm.endpoint(), + process.pid, + process.start_time, + deadline, + ) + .and_then(|transport| ControlSession::attach(transport, capability, deadline)) + .map_err(|error| self.session_error(error, control_timeout))?; + let request_id = session + .start_exec(workload, deadline) + .map_err(|error| self.session_error(error, control_timeout))?; + let timeout_ms = workload.timeout_ms; + let response_deadline = (timeout_ms > 0) + .then(|| Instant::now() + Duration::from_millis(timeout_ms.into()) + response_grace); + Ok(Execution { + session, + request_id, + deadline: response_deadline, + control_timeout, + }) + } +} + +#[cfg(test)] +mod tests; diff --git a/nvx-rs/src/host/instance/tests.rs b/nvx-rs/src/host/instance/tests.rs new file mode 100644 index 000000000..13921b740 --- /dev/null +++ b/nvx-rs/src/host/instance/tests.rs @@ -0,0 +1,215 @@ +use std::fs; + +use super::files::temporary_name; +use super::*; + +#[test] +fn interrupted_writes_leave_temporary_files_that_only_their_removal_takes() { + let directory = tempfile::tempdir().unwrap(); + let names = || { + let mut names: Vec = fs::read_dir(directory.path()) + .unwrap() + .map(|entry| entry.unwrap().file_name().into_string().unwrap()) + .collect(); + names.sort(); + names + }; + let path = directory.path().join("config.json"); + write_json(&path, &serde_json::json!({ "written": true })).unwrap(); + assert_eq!(names(), ["config.json"]); + // Replacements of the file that were interrupted left their temporary files, which other + // files only resemble. + let left = [0, 12345, u64::MAX].map(|number| temporary_name("config.json", number)); + let others = [ + ".config.json.tmp", + ".config.json..tmp", + ".config.json.012.tmp", + ".config.json.+12.tmp", + ".config.json.1a.tmp", + ".config.json.18446744073709551616.tmp", + ".config.json.1.tmp.tmp", + ".xconfig.json.1.tmp", + ".other.json.1.tmp", + "config.json.1.tmp", + ".tmpAbC123", + ]; + for name in left.iter().map(String::as_str).chain(others) { + fs::write(directory.path().join(name), b"{").unwrap(); + } + remove_interrupted_writes(&path).unwrap(); + let mut kept: Vec<&str> = others.into_iter().chain(["config.json"]).collect(); + kept.sort_unstable(); + assert_eq!(names(), kept); + // A directory that does not exist has none. + remove_interrupted_writes(&directory.path().join("gone").join("config.json")).unwrap(); +} + +#[test] +fn readiness_failures_explain_images_without_workload_accounts() { + let hint = "predates workload account creation"; + for ended in [SessionError::ProcessExited, SessionError::Closed] { + assert!(readiness_failure(&ended, true).contains(hint)); + // Only a sandbox that needs the account suggests it. + assert!(!readiness_failure(&ended, false).contains(hint)); + } + for other in [ + SessionError::TimedOut, + SessionError::Protocol("invalid record".to_owned()), + ] { + assert_eq!(readiness_failure(&other, true), other.to_string()); + } +} + +#[test] +fn session_failures_have_the_codes_of_their_kinds_but_a_closed_endpoint() { + assert_eq!( + session_code(&SessionError::Closed), + ErrorCode::EndpointClosed + ); + assert_eq!( + ErrorCode::EndpointClosed.kind(), + session_kind(&SessionError::Closed) + ); + for (error, code) in [ + (SessionError::TimedOut, ErrorCode::TimedOut), + (SessionError::ProcessExited, ErrorCode::Exited), + (SessionError::Reset, ErrorCode::Protocol), + (SessionError::AuthenticationFailed, ErrorCode::Protocol), + ( + SessionError::Protocol("bad".to_owned()), + ErrorCode::Protocol, + ), + (SessionError::Io(io::Error::other("lost")), ErrorCode::Io), + ] { + assert_eq!(session_code(&error), code, "{error}"); + } +} + +/// The instance's store, which exists on Linux and Windows hosts. +#[cfg(any(target_os = "linux", windows))] +mod store { + use std::fs; + + use serde_json::json; + + use super::super::*; + use crate::host::launch::PROCESS_COMMAND_LINE_LIMIT; + use crate::host::store::{ + ArtifactIdentity, JOURNAL_NAME, LOCK_NAME, LOG_NAME, RUNTIME_NAME, RemovalError, + RuntimeIdentity, + }; + + /// The configuration record of these tests' caller. + const CONFIG: &str = "config.json"; + + fn instance() -> (tempfile::TempDir, Instance) { + let directory = tempfile::tempdir().unwrap(); + let instance = Instance::create( + directory.path().join("vm"), + "test", + CallerFiles::new(CONFIG).unwrap(), + ) + .unwrap(); + (directory, instance) + } + + /// Returns the runtime identity of the launches of these tests. + fn runtime() -> RuntimeIdentity { + let artifact = |digit: &str| ArtifactIdentity::sha256(&digit.repeat(64)).unwrap(); + RuntimeIdentity::new(artifact("1"), artifact("2"), artifact("3")) + } + + #[test] + fn instances_that_record_no_vm_need_no_runtime_identity() { + let (_directory, instance) = instance(); + let mut locked = instance.lock(None).unwrap(); + let found = locked + .find(|| panic!("an instance that records no VM needs no runtime identity")) + .unwrap(); + assert!(matches!(found, Found::Stopped), "{found:?}"); + drop(locked); + // The instance's caller recorded no configuration, so its lock file went too. + assert!(!instance.dir().join(LOCK_NAME).exists()); + let reopened = + Instance::open(instance.dir(), instance.name(), instance.files().clone()).unwrap(); + assert_eq!(reopened, instance); + } + + #[test] + fn stopped_instances_are_removed_with_the_callers_files() { + let (_directory, instance) = instance(); + for name in [CONFIG, LOG_NAME, RUNTIME_NAME] { + fs::write(instance.dir().join(name), b"{}").unwrap(); + } + // An instance whose store records a VM is not removed. + let locked = instance.lock(None).unwrap(); + assert!(matches!(locked.remove(None), Err(RemovalError::Launched))); + assert!(instance.dir().join(CONFIG).is_file()); + fs::remove_file(instance.dir().join(RUNTIME_NAME)).unwrap(); + // Nor is one whose caller declared no configuration record. + let undeclared = + Instance::open(instance.dir(), instance.name(), CallerFiles::none()).unwrap(); + let locked = undeclared.lock(None).unwrap(); + assert!(matches!(locked.remove(None), Err(RemovalError::Undeclared))); + assert!(instance.dir().join(CONFIG).is_file()); + instance.lock(None).unwrap().remove(None).unwrap(); + assert!(!instance.dir().exists()); + // A transition that comes later finds the instance gone. + let error = instance.lock(None).unwrap_err(); + assert_eq!(error.kind(), ErrorKind::NotFound, "{error}"); + let error = + Instance::open(instance.dir(), instance.name(), CallerFiles::none()).unwrap_err(); + assert_eq!(error.kind(), ErrorKind::NotFound, "{error}"); + } + + #[test] + fn records_of_the_earlier_format_fail_naming_it() { + // Before the store, instances kept runtime records and launch markers of format 3. + for (name, record) in [ + ( + RUNTIME_NAME, + json!({ "format": 3, "pid": 7, "startTime": 8, "endpoint": "e" }), + ), + ( + JOURNAL_NAME, + json!({ "format": 3, "endpoint": "e", "process": { "pid": 7, "startTime": 8 } }), + ), + ] { + let (_directory, instance) = instance(); + let path = instance.dir().join(name); + fs::write(&path, record.to_string()).unwrap(); + let mut locked = instance.lock(None).unwrap(); + let error = locked.find(|| Ok(runtime())).unwrap_err(); + assert_eq!(error.kind(), ErrorKind::Io); + assert!( + error.message().contains("has the unsupported format 3"), + "{error}" + ); + assert!( + error + .message() + .contains("stop the sandbox with the build that started it"), + "{error}" + ); + // Nothing is migrated or removed. + assert!(path.is_file()); + } + } + + #[test] + fn launches_are_checked_before_the_instance_exists() { + let directory = tempfile::tempdir().unwrap(); + let mut spec = LaunchSpec::new( + Artifacts::new("openvmm", "vmlinux", "initramfs.cpio.gz"), + crate::Hypervisor::Kvm, + ); + let dir = directory.path().join("vm"); + check_launch(&dir, &spec).unwrap(); + spec.openvmm = directory + .path() + .join("o".repeat(PROCESS_COMMAND_LINE_LIMIT)); + let error = check_launch(&dir, &spec).unwrap_err(); + assert_eq!(error.kind(), ErrorKind::Unsupported, "{error}"); + assert!(!dir.exists()); + } +}