[Pedro Henrique Penna (ppenna)]

Fixes #568. Pins microsoft/openvmm#4804 at e4c741e1860de1e2149b3dd5d41863468705b02c, which makes the CPU checks of a microVM report every violation together, as section 2.2.1.3 of the architecture guide states. The E_CPU_SURFACE errors name every CPUID difference rather than the first 16, and the VM worker runs every CPU check of a partition and reports their failures in one error rather than stopping at the first. This pull request also maps the pin in the OpenVMM roadmap. The architecture guide does not change: OpenVMM now reports what it states, so no bound needs recording.

Based on dev at 8bee339dd. microsoft/openvmm#4804 is four commits directly on c32c57d12, the head of feature/microvm/dev and the revision that dev pins since #1016 merged, and the pin is that pull request's head commit itself, so this branch's two commits are its whole diff.

Change

Commit	Files	Change
545684c80 openvmm	openvmm	c32c57d12 → e4c741e18: the four commits of microsoft/openvmm#4804
f5f4c67bf doc	doc/openvmm-upstream-roadmap.md	Records them as positions 362-365, unassigned, with a remark that they change only the CPU checks of the VM worker's time ABI, which positions 159-269 added; moves the NVX pin; and records that microsoft/openvmm#4807 merged positions 360 and 361 into feature/microvm/dev

No other NVX change is needed. OpenVMM's control-session protocol vectors and built-in CPU profiles are unchanged at the pin. The line with which OpenVMM exits on these errors, such as fatal error: [E_CPU_SURFACE] failed to launch vm worker, keeps the code of the first check that fails, and nvx-rs attributes a clone's failure to an incompatible host from that code. The error's detail follows that line in OpenVMM's log. For the largest pinned profile, Turin, an error in which every register of all 94 entries differs in both comparisons and the backend supports nothing takes 61,717 bytes. That is within the 64 KiB at the end of the log that nvx-rs searches for the line; only a snapshot record that no OpenVMM writes, with most of the 1024 entries that the manifest decoder admits, could take more.

OpenVMM changes

• Registers of a recorded entry (69ddc715b). When the effective CPUID that a restore recomputes differs from the snapshot's record, the error names, for each entry that both hold, every register in which they differ, with its value and mask on each side, on one line. It printed all four results and masks of each side with {:#x?}, which pretty-prints each array over six lines: 24 lines per entry, without saying which register differs.
• Every difference (c55db5d9e). Both E_CPU_SURFACE errors, the restore's comparison with the snapshot's record and the comparison of VP 0's CPUID with the effective CPUID before any VP runs, name every difference rather than the first 16 (MAX_REPORTED_CPUID_DIFFERENCES), and keep counting them. The lists stay bounded: the effective CPUID of each pinned profile has between 64 entries (Skylake-SP) and 94 (Turin), and a snapshot's record at most 1024, which the manifest decoder enforces.
• Entries outside the profile with VP 0's differences (e1fa0f028). The worker checked VP 0's non-zero CPUID entries outside the profile's tables on a pass-through backend (E_CPU_UNLISTED) only once VP 0 observed every result, so a report that failed both named only the results. An E_CPU_SURFACE failure now names every result that VP 0 does not observe, then every such entry, as the backend support check names those entries after what a backend lacks, and a report whose only violations are such entries still fails with E_CPU_UNLISTED. No failure changes its code.
• Every check's failures together (e4c741e18). The worker ran its CPU checks in turn and stopped at the first that failed, so a restore whose record differed did not say whether VP 0 also read other values or the backend also lacked a feature of the profile. It now runs the comparison with the snapshot's record, the check of VP 0's CPUID, and the check that the backend supports the profile even after one fails. It reports their failures in one error that leads with the first, and so with the code that it had. A failure alone is reported as before.

The issue's tasks map onto this as follows:

• The limit. Removed, and the pin promoted, rather than recorded in section 2.2.1.3: every difference is named, and the effective CPUID's bounded size bounds the lists.
• The other verification steps. The backend support check names every feature, limit, XSAVE layout, address width, and MSR value that a backend lacks. The processor feature banks of MSHV and WHP name every feature that they lack, and the check of the entries outside the profile names every non-zero one. The generation check, and a restore's checks of the profile's ID, digest, and document, each have a single violation, and run before the partition exists, which the other checks need. Two gaps remained: the entries outside the profile went unreported behind a failed comparison of VP 0's results, which the third commit fixes, and the checks stopped at the first that failed, which the fourth fixes. MSHV and WHP refuse a profile feature that their processor feature banks lack when they create the partition, before these checks run. The backends' time ABI preflight checks the identity leaves, the CPU time bits, and, on WHP, the processor feature banks' hiding of the timer features. It enforces section 3.2.6's backend obligations rather than these steps, and those obligations fail fast.

Validation

• OpenVMM, on Windows: for openvmm_core, cargo clippy --all-targets -- -D warnings, cargo doc --no-deps, and cargo nextest run --profile agent at e4c741e18 (121 tests passed), with cpu_profile's 119, and cargo xtask fmt, all clean. Before [DO NOT REVIEW] microVM: name every CPUID difference in CPU-surface errors openvmm#4804 was rebased onto c32c57d12, its first three commits passed 118, 120, and 120 tests on bd2b27a2e; git range-diff shows the four commits unchanged by the rebase. The new tests fail with the limit restored, the test of the entries outside the profile fails if the error leaves them out, and the test of the combined checks fails if they stop at the first failure.

• End to end on the Azure test VMs nvx-kvm-test, nvx-mshv-test, and nvx-whp-test, at this branch's head, with the guest artifacts of release v0.1.0-dev.299d0b4d9851, whose OpenVMM, bd2b27a2e, served as the baseline:

  • Several failures. On Linux/KVM, a snapshot captured with the pin on the host profile intel.host.v1 (--cpu-profile host) had SSE4A (CPUID 0x80000001 ECX bit 6), which Intel CPUs lack, set in its manifest's profile document, and the document's digest updated. It fails to restore on each binary with the same exit line, fatal error: [E_CPU_SURFACE] failed to launch vm worker, and the pin names both failures:

    [E_CPU_SURFACE] the effective CPUID of CPU profile intel.host.v1 differs from the snapshot's in 1 entries: CPUID 0x80000001 ECX is 0x00000161 under mask 0xffffffff, and 0x00000121 under mask 0xffffffff in the snapshot; [E_PROFILE_UNSUPPORTED] the backend does not support CPU profile intel.host.v1: CPUID 0x80000001 ECX bit 6 is not supported
    

    [DO NOT REVIEW] microVM: name every CPUID difference in CPU-surface errors openvmm#4804's third commit as first pushed, 235da5cb8, names the first failure alone, and the baseline names it over 24 lines. MSHV and WHP refuse such a profile when they create the partition, before these checks run ([E_PROFILE_UNSUPPORTED] the hypervisor's processor features cannot present CPU profile intel.host.v1: they lack sse4a_support), with the pin as before.

  • Every difference. A snapshot captured with the pin, whose manifest's effective CPUID record then had bit 0 of EAX flipped in every entry, fails to restore with E_CPU_SURFACE on both binaries, with the same exit line. The baseline names 16 of the entries that it counts, over 321 lines. The pin names every one, on one line, and no other failure, as the other checks, which now run too, pass:

    Backend	Host CPU	CPU profile	Entries	Baseline names	Pin names
    Linux/KVM	Xeon Platinum 8573C	intel.emeraldrapids.v1	68	16	68 (7.4 KB)
    Linux/MSHV	Xeon Platinum 8573C	intel.emeraldrapids.v1	68	16	68 (7.4 KB)
    Windows/WHP	Xeon Platinum 8370C	intel.icelake-sp.v1	69	16	69 (7.5 KB)

• On the same VMs, python3 scripts/nvx.py test-microvm (and python scripts\nvx.py test-microvm on Windows) with the scenarios smp-snapshot (1, 2, 4, and 8 vCPUs), snapshot-core, and time-abi-conformance (8 vCPUs), whose cold boots and restores pass every CPU check, passed at the pin on Linux/KVM, Linux/MSHV, and Windows/WHP, as they did at 93cf08c4d, the pin before the rebase. A --x-time-abi-verify cold boot on each logs time ABI CPU checks passed with the checks' timings, as before.

• NVX at f5f4c67bf: python scripts\nvx.py verify (consistent); the openvmm_vectors and openvmm_cpu_profiles comparisons with --ignored (3 and 1 passed), against the pinned revision; python scripts\check_markdown_links.py; and git diff --check.

• Every commit is signed, both these and [DO NOT REVIEW] microVM: name every CPUID difference in CPU-surface errors openvmm#4804's.

• CI: before the rebase, [DO NOT REVIEW] microVM: name every CPUID difference in CPU-surface errors openvmm#4804's checks all passed at 93cf08c4d, and this pull request's at ca431309a. At e0f8b6ecc, on dev at 0eb0a27f2, NVX CLI / Linux failed in scripts/test_nvx_rs_mutants.py (MUTANTS_RESULT: unbound variable), as on other pull requests of that dev: ci: fix nvx-rs mutants summarize miscounting shards #952 changed the mutants workflow's merge step without its tests, which scripts: test the mutants merge against the shards' indexed outcomes #1023 fixed, so this branch now sits on dev with scripts: test the mutants merge against the shards' indexed outcomes #1023. At head f5f4c67bf, every check passed. The first attempt of run 40962952624 failed one runtime adapter scenario of NVX microVM tests / Linux / KVM / Guests and sandboxes, vm_clone's a_clone_refuses_a_scratch_image_whose_upper_an_earlier_clone_used, whose second clone accepted a used scratch image. That is the intermittent failure that runtime-adapter: a_clone_refuses_a_scratch_image_whose_upper_an_earlier_clone_used fails intermittently #1020 tracks, which dev's own push run 40949525134 also hit, at OpenVMM c32c57d12, and the rerun of the failed jobs passed. The pin changes no step of a CPU check that passes, so a clone's guest sees the same machine. On nvx-kvm-test, with that run's own guest artifacts and guest agent, vm_clone's 34 scenarios failed intermittently with both pins: in 6 of 14 runs with e4c741e18, 4 of them in that scenario, and in 2 of 11 runs with c32c57d12, in two other scenarios. [DO NOT REVIEW] microVM: name every CPUID difference in CPU-surface errors openvmm#4804's checks all passed at e4c741e18 too: the first attempt of its run 40958505253 failed the two verify openhcl binary size jobs, which compare with the feature/microvm/dev CI run of c32c57d12, then still running, and their rerun passed.

Merge microsoft/openvmm#4804 into feature/microvm/dev, by fast-forwarding that branch to e4c741e18, before merging this pull request.

[Copilot]

🟢 Approval recommended

No active review issues remain for the documentation update recording the OpenVMM pin and CPU-diagnostic commits.

No open issues 🎉

What changed in this PR

This PR promotes an OpenVMM revision that reports every CPUID difference in CPU-surface errors, addressing #568. The reviewed documentation maps those changes into NVX’s upstream roadmap.

Changes:

• Updates the documented NVX pin to 235da5cb8, position 362.
• Records the three CPU-diagnostic commits and their relationship to the existing time ABI work.
• Updates the commit counts, branch diagram, and merge history.

File	Description
doc/​openvmm-upstream-roadmap.md	Documents the new pin, positions 360–362, and their upstream context.

_{Review effort: Balanced}

[Copilot]

🟢 Approval recommended

No active review issues are identified for the roadmap update documenting CPU-verification diagnostics.

No open issues 🎉

_{Review effort: Balanced}

[Copilot]

🟢 Approval recommended

No active review issues are reported for the roadmap update documenting the CPU-verification diagnostic changes.

No open issues 🎉

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Integration depends on validation of the unmerged OpenVMM head, completion of queued exact-head CI run 40958505253, and confirmation of the required fast-forward into feature/microvm/dev.

No open issues 🎉

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

The OpenVMM gitlink pins an open PR with roughly 500 lines outside this diff, requiring maintainer confirmation of upstream source review and the prerequisite branch fast-forward and merge order.

No open issues 🎉

_{Review effort: Max}