nvx-rs: build a clone's activation record and launch its restore

nvx-rs/README.md

@@ -635,13 +635,27 @@ and no launch.
 the launch that the lease reserves, from metadata alone: the leased generation,
 its key, which must speak the current contracts, the boot and control contracts
 of the runtime that captured it included, and its snapshot; a network exactly
-when the template has a NIC; a memory target in whole blocks of
-`MEMORY_BLOCK_SIZE` from the template's RAM to its capacity; the clone's image,
-which the check of its `DistroBinding` opens; its upper, either the caller's
-scratch image, which the check of its `ScratchBinding` claims, and which nothing
-formats, copies, or reads, or for an upper in RAM a fresh sparse file,
-`clone-scratch.raw` in the sandbox's directory, which the release removes; and a
-fresh capability.
+when the template has a NIC, on the guest network of that NIC, whose address
+OpenVMM restores from the snapshot, and under the policy of that NIC, which
+OpenVMM checks as it restores it, since the key names the policy only by
+OpenVMM's identity of it; a memory target in whole blocks of `MEMORY_BLOCK_SIZE`
+from the template's RAM to its capacity; the clone's image, which the check of
+its `DistroBinding` opens; its upper, either the caller's scratch image, which
+the check of its `ScratchBinding` claims, and which nothing formats, copies, or
+reads, or for an upper in RAM a fresh sparse file, `clone-scratch.raw` in the
+sandbox's directory, which the release removes; and a fresh capability.
+
+It builds the clone's activation record, `ClonePlan::activation`, which the
+clone's guest applies in place of the configuration that the template captured,
+from the request's `WorkloadView`, overlay mode, and journal policy, and from
+the checked inputs: a restore section with a fresh transaction ID of 16 random
+bytes and the lease's `TemplateReference`; the image, an EROFS layer, with its
+binding's identity and the UUID that the adapter's plan expects of it, since a
+GPT image is refused; a tmpfs, or the scratch image's binding identity with the
+journal policy; no export; the template NIC's policy identity, as its key
+records it, unpublished listeners, and the guest's proxy URL; and the template's
+processors and the memory target. A binding's identity is the SHA-256 digest of
+a domain and of the binding's serde form, never of a file's content.
 
 The image and the scratch device must present the `BlockGeometry` of the
 template's, which a restore compares, and be, at the paths by which OpenVMM
@@ -652,6 +666,34 @@ scratch image may be a file that the sandbox's store owns, such as
 transitions replace or remove them, and a link or a directory under one of the
 store's names is refused.
 
+`ClonePlan::launch` then launches the clone as the launch that its lease
+reserves, once it has checked that the sandbox's clone binding records the
+lease, and returns a `Restoring`: it writes the clone's activation record in its
+canonical encoding, `clone-activation.bin`, from which a caller resends it
+unchanged; it listens for the VMM's restore-ready event, at the socket
+`restore.sock` on Linux, and at a named pipe of a random name on Windows; and it
+runs `supervisor::launch` with OpenVMM's restore of the clone's devices,
+`--restore-snapshot` with the generation's snapshot in place, which no clone
+copies or changes, so that clones of one generation restore it at once,
+`--restore-memory` with the memory target, and `--restore-ready-path`, under the
+template's backend, CPU profile, and processors, with a boot console that
+discards its output, as cold boots have, and the control console, but no kernel,
+initramfs, command line, or guest address, which the snapshot fixes.
+
+On Linux, where OpenVMM requires a restore's control socket in the snapshot's
+parent, the clone's VMM listens in the generation's payload, at `<launch>.sock`,
+named after the launch that the lease reserves, whose path must be shorter than
+108 bytes with or without the payload's links resolved; on Windows, it listens
+at a named pipe, as every launch does. The VMM inherits the claim of a scratch
+image, on Windows through the image's guard, once the launch has closed the
+claim's handle, which OpenVMM's open of the image would refuse.
+
+A payload holds the sockets of its clones only while they lease it: the VMM
+removes its socket as it exits, a stop or a recovery removes the one that an
+abrupt exit leaves, and the release makes the socket's absence durable before
+the lease ends, and removes the record and `restore.sock`, as it removes
+`clone-scratch.raw`.
+
 `Store::tombstone` quarantines a generation, which takes no new lease while its
 leases keep working, and `Store::retire` deletes it once its leases have
 drained, crash-safely, removing the records of leases that no clone binding

nvx-rs/src/host/platform/linux.rs

@@ -2,7 +2,7 @@ use std::fs::{self, DirBuilder, File, OpenOptions, Permissions, TryLockError};
 use std::io::{self, Read};
 use std::os::fd::{AsFd, AsRawFd, RawFd};
 use std::os::unix::fs::{DirBuilderExt, MetadataExt, OpenOptionsExt, PermissionsExt};
-use std::os::unix::net::UnixStream;
+use std::os::unix::net::{UnixListener, UnixStream};
 use std::path::Path;
 use std::process::Command;
 use std::thread;
@@ -639,6 +639,25 @@ pub(crate) fn opened_file_identity(file: &File) -> io::Result<(u64, u64)> {
     Ok((metadata.dev(), metadata.ino()))
 }
 
+/// Returns the identity by which the stores tell a file or directory apart from every other,
+/// following its links: its device and inode, as [`opened_file_id`] gives them.
+pub(crate) fn file_id(path: &Path) -> io::Result<(u64, u128)> {
+    let metadata = fs::metadata(path)?;
+    Ok((metadata.dev(), u128::from(metadata.ino())))
+}
+
+/// Returns the identity by which the stores tell the file that `file` holds open apart from every
+/// other: its device and inode, as its [`FileSeal`](crate::host::seal::FileSeal) holds them.
+pub(crate) fn opened_file_id(file: &File) -> io::Result<(u64, u128)> {
+    let metadata = file.metadata()?;
+    Ok((metadata.dev(), u128::from(metadata.ino())))
+}
+
+/// Returns the number of names, its hard links, of the file that `file` holds open.
+pub(crate) fn link_count(file: &File) -> io::Result<u64> {
+    Ok(file.metadata()?.nlink())
+}
+
 /// An open directory, through which operations name its entries, so that they reach the entries
 /// of the directory that it opened, whatever its path names meanwhile.
 pub(crate) struct Directory(File);
@@ -655,19 +674,102 @@ impl Directory {
             .map(Self)
     }
 
-    /// Returns the device and inode of its entry `name`, not following a link, as
-    /// [`opened_file_identity`] gives them, or `None` if it has no such entry.
-    pub(crate) fn entry_identity(&self, name: &str) -> io::Result<Option<(u64, u64)>> {
+    /// Returns the identity of its entry `name`, not following a link, as [`opened_file_id`] gives
+    /// it, or `None` if it has no such entry.
+    pub(crate) fn entry_id(&self, name: &str) -> io::Result<Option<(u64, u128)>> {
         use nix::fcntl::AtFlags;
         use nix::sys::stat::fstatat;
 
         match fstatat(&self.0, name, AtFlags::AT_SYMLINK_NOFOLLOW) {
-            Ok(status) => Ok(Some((status.st_dev, status.st_ino))),
+            Ok(status) => Ok(Some((status.st_dev, u128::from(status.st_ino)))),
             Err(Errno::ENOENT) => Ok(None),
             Err(errno) => Err(errno.into()),
         }
     }
 
+    /// Opens its entry `name` for reading, not following a link, and without waiting, as an open
+    /// of a FIFO without a writer would, or returns `None` if it has no such entry. The caller
+    /// checks what the entry is.
+    pub(crate) fn open_file(&self, name: &str) -> io::Result<Option<File>> {
+        use nix::fcntl::{OFlag, openat};
+        use nix::sys::stat::Mode;
+
+        match openat(
+            &self.0,
+            name,
+            OFlag::O_RDONLY | OFlag::O_NOFOLLOW | OFlag::O_NONBLOCK | OFlag::O_CLOEXEC,
+            Mode::empty(),
+        ) {
+            Ok(file) => Ok(Some(File::from(file))),
+            Err(Errno::ENOENT) => Ok(None),
+            Err(errno) => Err(errno.into()),
+        }
+    }
+
+    /// Opens the directory `path` with the right to search it alone, without following a link at
+    /// its own name, though links on the way to it are followed, or returns `None` if it does not
+    /// exist. Fails if it is a link, or not a directory.
+    ///
+    /// The handle names the directory's entries, and needs no right to read the directory, which
+    /// an owner-only directory may withhold; it cannot [sync](Self::sync) it.
+    pub(crate) fn search(path: &Path) -> io::Result<Option<Self>> {
+        use nix::fcntl::OFlag;
+
+        match OpenOptions::new()
+            .read(true)
+            .custom_flags((OFlag::O_PATH | OFlag::O_DIRECTORY | OFlag::O_NOFOLLOW).bits())
+            .open(path)
+        {
+            Ok(dir) => Ok(Some(Self(dir))),
+            Err(error) if error.kind() == io::ErrorKind::NotFound => Ok(None),
+            Err(error)
+                if [Errno::ELOOP, Errno::ENOTDIR]
+                    .map(|errno| Some(errno as i32))
+                    .contains(&error.raw_os_error()) =>
+            {
+                Err(io::Error::other(format!(
+                    "{} is a link, or not a directory",
+                    path.display()
+                )))
+            }
+            Err(error) => Err(error),
+        }
+    }
+
+    /// Opens its subdirectory `name` with the right to search it alone, not following a link, as
+    /// [`search`](Self::search) opens a directory, or returns `None` if it has no such entry.
+    /// Fails if the entry is a link, or not a directory.
+    pub(crate) fn search_dir(&self, name: &str) -> io::Result<Option<Self>> {
+        self.open_dir_with(name, nix::fcntl::OFlag::O_PATH)
+    }
+
+    /// Opens its subdirectory `name` for reading, not following a link, or returns `None` if it
+    /// has no such entry. Fails if the entry is a link, or not a directory.
+    pub(crate) fn open_dir(&self, name: &str) -> io::Result<Option<Self>> {
+        self.open_dir_with(name, nix::fcntl::OFlag::O_RDONLY)
+    }
+
+    /// Opens its subdirectory `name` with the access `access`, not following a link, as
+    /// [`open_dir`](Self::open_dir) and [`search_dir`](Self::search_dir) do.
+    fn open_dir_with(&self, name: &str, access: nix::fcntl::OFlag) -> io::Result<Option<Self>> {
+        use nix::fcntl::{OFlag, openat};
+        use nix::sys::stat::Mode;
+
+        match openat(
+            &self.0,
+            name,
+            access | OFlag::O_DIRECTORY | OFlag::O_NOFOLLOW | OFlag::O_CLOEXEC,
+            Mode::empty(),
+        ) {
+            Ok(dir) => Ok(Some(Self(File::from(dir)))),
+            Err(Errno::ENOENT) => Ok(None),
+            Err(Errno::ELOOP | Errno::ENOTDIR) => Err(io::Error::other(format!(
+                "{name} is a link, or not a directory"
+            ))),
+            Err(errno) => Err(errno.into()),
+        }
+    }
+
     /// Removes its file `name`, if it has one.
     pub(crate) fn remove_file(&self, name: &str) -> io::Result<()> {
         use nix::unistd::{UnlinkatFlags, unlinkat};
@@ -753,6 +855,31 @@ pub(crate) fn control_endpoint(socket_path: &Path) -> io::Result<String> {
         .ok_or_else(|| io::Error::other("control socket path is not valid UTF-8"))
 }
 
+/// The listener at which the launch of a clone awaits the restore-ready event of its VMM: a Unix
+/// socket, which the VMM connects to once, and which queues its connection until the host
+/// accepts it.
+#[derive(Debug)]
+pub(crate) struct ReadyListener {
+    /// The listening socket, which the VMM finds as long as it is open.
+    _socket: UnixListener,
+}
+
+/// Listens at the Unix socket `socket_path`, in place of a socket that an earlier launch left
+/// there, as the restore-ready endpoint of a clone's launch, and returns the endpoint, as
+/// OpenVMM's `--restore-ready-path` takes it, and its listener.
+pub(crate) fn listen_ready(socket_path: &Path) -> io::Result<(String, ReadyListener)> {
+    let endpoint = socket_path
+        .to_str()
+        .map(str::to_owned)
+        .ok_or_else(|| io::Error::other("restore-ready socket path is not valid UTF-8"))?;
+    match fs::remove_file(socket_path) {
+        Err(error) if error.kind() != io::ErrorKind::NotFound => return Err(error),
+        _ => {}
+    }
+    let socket = UnixListener::bind(socket_path)?;
+    Ok((endpoint, ReadyListener { _socket: socket }))
+}
+
 /// Connects to a control endpoint served by the process `expected_pid`.
 ///
 /// The connection attempt never blocks: a listener whose backlog is full yields

nvx-rs/src/host/platform/mod.rs

@@ -171,4 +171,30 @@ mod tests {
             Err(error) => panic!("{error}"),
         }
     }
+
+    /// The stores tell files apart by the identities that their seals hold, with the 128-bit file
+    /// IDs of `FILE_ID_INFO` on Windows, whether they examine a path, an open file, or an entry of
+    /// an open directory.
+    #[cfg(any(target_os = "linux", windows))]
+    #[test]
+    fn the_stores_tell_files_apart_by_the_identities_that_their_seals_hold() {
+        use crate::host::seal::FileSeal;
+
+        let directory = tempfile::tempdir().unwrap();
+        let opened = Directory::open(directory.path()).unwrap();
+        let mut ids = Vec::new();
+        for name in ["first", "second"] {
+            let path = directory.path().join(name);
+            std::fs::write(&path, b"").unwrap();
+            let file = std::fs::File::open(&path).unwrap();
+            let seal = FileSeal::of(&file).unwrap();
+            let id = (seal.device(), seal.file_id());
+            assert_eq!(opened_file_id(&file).unwrap(), id, "{name}");
+            assert_eq!(file_id(&path).unwrap(), id, "{name}");
+            assert_eq!(opened.entry_id(name).unwrap(), Some(id), "{name}");
+            ids.push(id);
+        }
+        assert_ne!(ids[0], ids[1]);
+        assert_eq!(opened.entry_id("missing").unwrap(), None);
+    }
 }

nvx-rs/src/host/platform/other.rs

@@ -98,6 +98,20 @@ pub(crate) fn opened_file_identity(_file: &File) -> io::Result<(u64, u64)> {
     Err(unsupported())
 }
 
+/// Fails: this host keeps no store.
+pub(crate) fn file_id(_path: &Path) -> io::Result<(u64, u128)> {
+    Err(unsupported())
+}
+
+/// Fails: this host keeps no store.
+pub(crate) fn opened_file_id(_file: &File) -> io::Result<(u64, u128)> {
+    Err(unsupported())
+}
+
+pub(crate) fn link_count(_file: &File) -> io::Result<u64> {
+    Err(unsupported())
+}
+
 /// An open directory, which this host never holds: it keeps no sandbox's store.
 pub(crate) enum Directory {}
 
@@ -108,7 +122,27 @@ impl Directory {
     }
 
     /// Never called: no directory is open.
-    pub(crate) fn entry_identity(&self, _name: &str) -> io::Result<Option<(u64, u64)>> {
+    pub(crate) fn entry_id(&self, _name: &str) -> io::Result<Option<(u64, u128)>> {
+        match *self {}
+    }
+
+    /// Never called: no directory is open.
+    pub(crate) fn open_file(&self, _name: &str) -> io::Result<Option<File>> {
+        match *self {}
+    }
+
+    /// Never called: no directory is open.
+    pub(crate) fn open_dir(&self, _name: &str) -> io::Result<Option<Self>> {
+        match *self {}
+    }
+
+    /// Fails: this host keeps no store.
+    pub(crate) fn search(_path: &Path) -> io::Result<Option<Self>> {
+        Err(unsupported())
+    }
+
+    /// Never called: no directory is open.
+    pub(crate) fn search_dir(&self, _name: &str) -> io::Result<Option<Self>> {
         match *self {}
     }
 
@@ -210,6 +244,14 @@ pub(crate) fn control_endpoint(_socket_path: &Path) -> io::Result<String> {
     Err(unsupported())
 }
 
+/// The listener of the restore-ready endpoint of a clone's launch, which this host never makes.
+#[derive(Debug)]
+pub(crate) struct ReadyListener;
+
+pub(crate) fn listen_ready(_socket_path: &Path) -> io::Result<(String, ReadyListener)> {
+    Err(unsupported())
+}
+
 pub(crate) fn connect_endpoint(
     _endpoint: &str,
     _expected_pid: u32,