nvx-rs: restore clones from a template through Activate, Ready and Start #844

Parent: #834 (#833), step 3. Needs: #841, #843, and, in the test adapter, #822's PRs 2 and 7 and #805's PR 3. Unblocks: #845, #846.

Goal. Restore a clone from a leased generation, with its own image, upper and memory target. Connect after OpenVMM's restore-ready event, drive Activate, Ready and Start, classify failures, tear down, and reattach.

Today (dev d04d9fb):

• nvx-rs run restores a raw snapshot on the terminal. It renders --restore-snapshot, --restore-processors, --restore-memory and --restore-ready-path (nvx-rs/src/host/cli/run.rs:72-75, :383-392). The library has no restore.
• The host attaches with session::connect and ControlSession::attach, pings for APP_READY, then sends MAPS and EXEC (nvx-rs/src/host/session.rs:77-223).
• ScratchBinding::check claims a scratch disk until the VMM releases it (sandbox/scratch.rs:110-230), and DistroBinding checks an image by metadata (sandbox/distro.rs:162-230). Neither records the block geometry.
• supervisor::reattach returns Reattached::Mismatched for another runtime. Vmm::stop and Vmm::terminate confirm the exit through the verified process (supervisor/mod.rs:391-705).

Stacked PRs. Each PR builds on the one before it, and lands with its own tests.

PR 1 of 5: the clone plan and block geometry. Needs #839's PR 3 and #841's PR 2.

• ClonePlan holds:
  • the leased generation;
  • the image, through DistroBinding. In this stage it must be exactly the placeholder's length;
  • the upper: for scratch, a caller-supplied scratch file, claimed with ScratchBinding, whose identity, kind, length, geometry and exclusive claim are checked, and which is never formatted, copied or read; for tmpfs, a sparse file of the scratch's length in the clone's private directory, removed at teardown;
  • a memory target M0 <= M <= Cmem, in 128 MiB blocks;
  • fresh boot-console and control endpoints, and a fresh capability.
• DistroBinding and ScratchBinding gain the block geometry that OpenVMM will present, read through the checked handle as OpenVMM reads it: the length (nonzero, 512-byte aligned), the logical block size (512), and the physical block size, which is the file's st_blksize on Linux and 4096 on Windows (openvmm/openvmm_entry/src/storage_builder/microvm.rs:247-262).
• A request that the template can't serve, such as another length or geometry, fails here, before launch.
• Tests: length and geometry refusals, including a file on a filesystem with another block size; targets out of range; a scratch claim's identity, kind and exclusivity.

PR 2 of 5: the activation record and the restore launch. Needs PR 1, #838's PR 3, and #839's PR 4.

• Before launch, the host builds the ActivationRecord from the checked inputs:

  • the restore section: a fresh transaction ID of 16 random bytes, and the generation and the key's identity;
  • the image: its format, DistroBinding's identity, and the expected EROFS UUID or GPT disk GUID, if the adapter gives one;
  • the upper: tmpfs, or ScratchBinding's identity;
  • the workload view and the root settings, from the adapter's request and the template's invariants;
  • the expected resources, from the targets;
  • no export, and only the template's network settings, until nvx-rs: exports, MAPS and network templates for clones #846.

  Ready must report that the activation applied, with the record's transaction ID, as ReadyActivation::of (nvx::activation) gives it.

• Once the lease and the clone binding are durable (nvx-rs: the template compatibility key, the template store and leases #839's seam), supervisor::launch starts OpenVMM's restore with the image as distro, the scratch file, the template's network policy, --restore-memory and --restore-ready-path.

• Tests, with a fake VMM: the record's contents; the rendered restore arguments; no spawn before the binding is durable; crashes around spawn.

PR 3 of 5: the restore sequence. Needs PR 2, and #843's PR 3.

1. Wait for the restore-ready event, under the host's restore-to-Ready deadline, which starts at the launch.
2. Connect with the fresh capability.
3. Send Activate, with the transaction ID, through the adapter's framing.
4. Check that Ready reports the activation applied, with the transaction ID, and the adapter's identities.
5. Start, under a Ready-to-Start deadline.

• Retries resend the same transaction and a byte-equal record. The default deadlines come from the slowest backend's CI runs.
• Tests, with a fake VMM and transport: wrong Ready identities; the bootstrap against a later reset; no request before Activate; identical and divergent retries; deadlines; crashes around Ready and Start, each recovered with the right lease state.

PR 4 of 5: failures, teardown and reattach. Needs PR 3.

• Typed outcomes, attributed by evidence:

  Evidence	Outcome	Quarantine
  Authorization denied; a stale image binding; invalid targets, format, settings, Activate or mappings; an image that the template can't serve; a scratch that isn't ext4 or isn't fresh	Request	No
  A supplied file with the wrong identity, length or geometry	Request, or host staging when proven	No
  An endpoint or forward bind failure; resource exhaustion; an incompatible destination CPU or TSC	Host	No
  Lost status; a timeout; a reset; a forced kill	Host, protocol or unknown	No, unless stronger evidence shows the provenance
  A malformed or inconsistent generation, or a diagnosed capture or repair defect attributable to it	Template	Yes, through that generation's tombstone
  A repair status alone, including 193, 194 and 195	Repair stage; provenance still to establish	No

• The records keep the stage, the cause, the transaction, the generation, the acknowledgment and activation progress, the original outcome and the teardown outcome. A launch keeps its generation until the VM exits, so a late failure after Ready is still classified.

• Teardown: a failure before spawn has no VM. Later, the verified VM is stopped and its exit confirmed. An indeterminate identity or exit returns the failure with teardown unresolved, and keeps the records and the lease. There is no cold fallback.

• Reattach: the process identity says which VM it is, the runtime identity says whether this host may manage it, and the template reference keeps the lease, which a tombstone or retirement doesn't block. Reattach checks the established activation and transaction on a new session. A runtime mismatch stops the clone by verified identity.

• Tests, with fakes: each evidence row's outcome; verified teardown, and leases kept when it's uncertain; reattach, and a runtime mismatch.

PR 5 of 5: clones in VMs through the test adapter. Needs PR 4, #840's PR 2, #842's PR 3, #843's PR 3, #822's PRs 2 and 7, and #805's PR 3.

• The test adapter gets a path that starts a clone from a template; runtime adapter: template and clone operations #847 completes its operations.
• Tests, in MicroVM CI on KVM, MSHV and WHP:
  • one generation reused across uppers and memory targets;
  • images of other lengths refused before launch; a geometry mismatch; a scratch that isn't ext4, refused as a request fault;
  • restarts with reattach; a runtime mismatch; a leased clone of a tombstoned generation;
  • same-boot and reboot eligibility, with clock fixtures;
  • no large-content read on restore, scratch included, with a positive control;
  • a record with a workload identity, which executions run as, and one whose account the image lacks, refused before Ready;
  • nvx-rs: repair a restored clone in the guest, with an owned seed and one acknowledgment #842's and nvx-rs: the restricted control state and Activate after a restore, in the guest library and the test adapter's framing #843's VM cases: memory targets, deferred time work after Ready, no host input before the acknowledgment, and no workload before Ready.

Not in scope: exports, MAPS and network templates (#846); GPT clones and the fresh-scratch check (#845); the test adapter's full operations (#847).

Guide (#591's numbering): 2.1.4 (restore-time configuration), 3.4.4 (the image at the placeholder's length), 5.2.4 (a clone's lifecycle), 5.4.7 (attachments at restore), 5.5.1 (the restore sequence), 5.5.2 and 5.5.4 (scratch and memory), 5.6.2 (failure classes).

Done when: the five PRs are merged, and EROFS clones over tmpfs and scratch run through Start in CI on the three backends.