[DO NOT REVIEW] cpu_profile: verify pinned CPU profiles at partition creation by ppenna · Pull Request #4807 · microsoft/openvmm

Pedro Henrique Penna (ppenna) · 2026-10-29T18:13:54Z

Verifies each pinned CPU profile before a partition uses it, for microsoft/nvx#567: once per process, on its first use, the catalog checks that the profile built from pinned_data.rs is valid, with its density, ownership masks, and CPU time bits, and that its constant encoding, which snapshots embed, is its canonical encoding. Profile validation also gains the CPU time bits, so the host profile document that a snapshot carries must fix them too.

Based on feature/microvm/dev. This branch is two commits directly on bd2b27a2e, the head of feature/microvm/dev since #4794 merged, and the revision that NVX's dev pins.

Why

At partition creation, for a cold boot and a restore, the NVX time ABI requires the VMM to verify that the CPU profile is valid, including its canonical encoding, density, ownership masks, and CPU time bits (step 1 of the CPU profile verification in section 2.2.1.3 of NVX's architecture guide). OpenVMM did so for host profiles, which derivation validates and which a restore decodes from the snapshot's canonical document, but not for pinned profiles: catalog::load built a pinned profile from its constants on first use with "no parsing, no validation, and no hashing, which the tests do for every pinned profile instead", and pinned_for_restore compared a snapshot's digest with a constant that only the tests checked. A stale or malformed pinned_data.rs therefore reached partition creation unchecked.

CpuProfile::validate never checked the CPU time bits either. Derivation applies them, and the tests check the pinned profiles' effective CPUIDs, but a host profile document that a snapshot carries was decoded and accepted without them: only the WHP preflight checks the time bits that the backend presents.

CPU time bits (`c9ac93a63`)

CpuProfile::validate checks every CPU time bit, from a table of the bits that each leaf's registers must hold (profile::TIME_BITS): the hypervisor, TSC, and RDTSCP bits set; ARAT alone in leaf 0x6 and the invariant TSC alone in leaf 0x80000007; no TSC-deadline timer, PDCM, or IA32_TSC_ADJUST; and leaves 0xa, 0x15, and 0x16 zero. A leaf that a profile does not list reads as zero, so leaves 0x15 and 0x16 above the maximum basic leaf comply. The check names every violation by leaf, subleaf, register, and bit, as support_violations does, and fails like any other invariant: decoding a host profile document that violates the time bits, and so restoring it, fails with E_PROFILE_DIGEST. Every pinned and derived profile complies, so no profile changes.

Pinned profile verification (`c32c57d12`)

PinnedProfile::verify checks a pinned profile's static data: the profile that its constants build is valid (CpuProfile::validate, time bits included), and its constant encoding is that profile's canonical encoding. A violation fails with E_PROFILE_DIGEST and names the profile and what is wrong, for example pinned CPU profile intel.icelake-sp.v1 is invalid: it violates the CPU time bits: CPUID 0x1 ECX bit 24 is set, or pinned CPU profile intel.icelake-sp.v1 differs from its constant canonical encoding.

Verification hashes nothing. The constant digest is the profile's identity, which the tests keep checking against the golden digests, as they keep checking the static data against the reviewed JSON files, so a pinned profile's start still neither parses JSON nor hashes. A restore compares the snapshot's digest with that constant and its document with the verified encoding.

The catalog verifies each profile once per process and hands out only verified ones:

load caches the verification next to the built profile, so each process verifies each profile that it uses once.
pinned, pinned_for_restore, select, and PartitionProfile::pinned fail with E_PROFILE_DIGEST for an invalid profile. pinned and PartitionProfile::pinned return a Result instead of an Option, with E_PROFILE_UNKNOWN for an unknown ID. A PartitionProfile holds the verified profile, so dereferencing it stays infallible.
pinned_for_restore verifies the profile before it compares the snapshot's digest, so a defective build does not report a mismatched snapshot.
select_auto still builds every revision that covers the host, to select by ID and generation, but verifies only the one that it selects: a Genoa host, which two revisions cover, verifies one.
pinned_profiles, for tools and tests, panics on an invalid profile.

Cost

Verification adds to the first use of a profile in a process, which NVX's single-process OpenVMM makes once per cold boot or restore. Medians of the first select or pinned_for_restore in fresh processes of a release build, for the Skylake-SP, Ice Lake-SP, and Genoa profiles (the "warmed" runs busy-wait 100 ms first, so that the clock is up, as it is after the work that precedes selection in a starting OpenVMM):

Host	CPU	Before	After	Warmed, before	Warmed, after
Windows laptop	Core i9-12900H	8-19 us	50-69 us	10-27 us	54-86 us
Azure VM, Windows	Xeon Platinum 8370C	14-31 us	68-84 us	18-37 us	75-91 us
Bare metal, Linux	Xeon Silver 4114 (Skylake-SP)	8-27 us	158-170 us	2-8 us	45-49 us
Bare metal, Windows	Xeon Silver 4114 (Skylake-SP)	24-66 us	129-224 us	17-69 us	138-229 us

Validation and the encoding take about a third and two thirds of it. Also checking the digest would have hashed the 9 KB encoding at every start, which nearly doubles the cost on CPUs without the SHA extensions, such as Skylake-SP; the measured restores and cold boots of NVX take 36-200 ms and 275-720 ms.

Tests

profile::tests::validation_rejects_profiles_that_violate_the_cpu_time_bits: every pinned profile fixes the time bits, those whose maximum basic leaf is below 0x15 included, and each time bit that an edited document sets or clears fails its decoding with the bit named, all of them together when several do.
catalog::tests::restore_decodes_and_checks_a_host_profile: a canonical host profile document with its own digest but a TSC-deadline timer fails its restore.
catalog::tests::verification_rejects_stale_or_malformed_static_data: static data that violates the time bits, misses a leaf, pins a VM-owned field, names another generation, or whose encoding does not hold its values, as after a hand edit, fails verification with the problem named; a digest that is not the encoding's does not, since the tests own it.
catalog::tests::the_static_data_is_the_pinned_files now also verifies each pinned profile and checks that loading it recorded the verification.

Validation

On Windows with Rust 1.95.0: cargo clippy --all-targets -- -D warnings for cpu_profile, virt, virt_whp, openvmm_helpers, openvmm_core, hypervisor_resources, openvmm_hypervisors, openvmm_entry, and petri; cargo doc --no-deps -p cpu_profile; cargo nextest run --profile agent -p cpu_profile (119 passed); and cargo xtask fmt --fix (clean).
On Linux with Rust 1.95.0, at this head's tree: cargo clippy --all-targets -- -D warnings for the same crates and virt_kvm and virt_mshv (clean), and cargo nextest run --profile agent for cpu_profile, virt, virt_kvm, virt_mshv, openvmm_helpers, and openvmm_core (541 passed, 26 skipped).
NVX's default microVM suite with this branch's tree, run by test-microvm after build-openvmm with the guest artifacts of NVX's v0.1.0-dev.299d0b4d9851. Every scenario cold boots on the host's pinned profile (--cpu-profile auto), many capture and restore, and one captures and restores a host profile. Each run executed 47 scenarios, 41 distinct ones, and all passed: Linux/KVM on a Xeon Silver 4114 bare-metal host (intel.skylake-sp.v1), at this head through openvmm: verify pinned CPU profiles at partition creation nvx#1016, and on a Xeon Platinum 8573C Azure VM (intel.emeraldrapids.v1); Linux/MSHV on a Xeon Silver 4114 bare-metal host (intel.skylake-sp.v1); and Windows/WHP on a Xeon Platinum 8370C Azure VM (intel.icelake-sp.v1), whose Windows unit tests of cpu_profile, virt, virt_whp, openvmm_helpers, and openvmm_core passed too (460 passed, 18 skipped). The runs other than the first used 0565b55bc, which differs from this head only in its commit message (tree 96769697845a).
Defective built-in data, built into this branch and into bd2b27a2e on the KVM Azure VM. A good build captured a snapshot, which the defective build then restored, and the defective build cold booted. With the TSC-deadline bit set in the values of intel.emeraldrapids.v1 but not in its encoding, this branch refuses the restore and the cold boot within 0.01 s with [E_PROFILE_DIGEST] pinned CPU profile intel.emeraldrapids.v1 is invalid: it violates the CPU time bits: CPUID 0x1 ECX bit 24 is set; bd2b27a2e fails only at the backend support check, with E_PROFILE_UNSUPPORTED. With ERMS cleared in the values but not in the encoding, which stays a profile that every backend supports, this branch refuses both with [E_PROFILE_DIGEST] pinned CPU profile intel.emeraldrapids.v1 differs from its constant canonical encoding, while bd2b27a2e boots and snapshots a guest without ERMS whose snapshots record a profile with it.
openvmm: verify pinned CPU profiles at partition creation nvx#1016, which pins this head, passed NVX's CI, including its microVM suites on KVM, MSHV, and WHP, and its OpenVMM unit and VMM tests.

Part of microsoft/nvx#567.

The NVX time ABI fixes a set of CPU time bits in every CPU profile, and its CPU profile verification at partition creation requires a valid profile, CPU time bits included. CpuProfile::validate checked a profile's identity, density, ownership masks, XSAVE layout, MSRs, and address width, but not the time bits. Derivation applies them, so the host profiles that a cold boot derives comply, and the catalog's tests check that the pinned profiles' effective CPUIDs do, but the host profile that a snapshot carries was decoded and accepted without them: only the WHP preflight checks the time bits that the backend presents, and the KVM and MSHV preflights do not. Validation now checks every time bit: the hypervisor, TSC, and RDTSCP bits set; ARAT alone in leaf 6 and the invariant TSC alone in leaf 0x80000007; no TSC-deadline timer, PDCM, or IA32_TSC_ADJUST; and leaves 0xa, 0x15, and 0x16 zero. A leaf that a profile does not list reads as zero, so leaves 0x15 and 0x16 above the maximum basic leaf comply. The check names every violation by leaf, subleaf, register, and bit, and fails like any other invariant, so a host profile document that violates the time bits fails to decode, and its restore fails, with E_PROFILE_DIGEST. Every pinned and derived profile complies, so no profile changes. The Guide's time and CPU compatibility section lists the time bits. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

At partition creation, for a cold boot and a restore, the NVX time ABI requires the VMM to verify that the CPU profile is valid: its canonical encoding, density, ownership masks, and CPU time bits. OpenVMM did so for host profiles, which derivation validates and which a restore decodes from the snapshot's canonical document, but not for pinned profiles: the catalog built a pinned profile from the constants in pinned_data.rs on first use without validating or encoding it, and only the catalog's tests checked those constants. A stale or malformed pinned_data.rs therefore reached partition creation unchecked. PinnedProfile::verify now checks the static data of a pinned profile: the profile is valid, with its density, ownership masks, and CPU time bits, and its constant encoding, which snapshot records embed and which a restore compares with the snapshot's document, is its canonical encoding. A violation fails with E_PROFILE_DIGEST and names the profile and what is wrong. Verification encodes the profile but hashes nothing: the constant digest is the profile's identity, which the tests keep checking against the golden digests, as they keep checking the static data against the reviewed JSON files, so no start path hashes, as before. The catalog verifies each pinned profile once per process, on its first use, and hands out only verified profiles: pinned, pinned_for_restore, select, and PartitionProfile::pinned fail with E_PROFILE_DIGEST for an invalid one, so pinned and PartitionProfile::pinned return a Result instead of an Option, and report an unknown ID with E_PROFILE_UNKNOWN. A PartitionProfile holds the verified profile, so dereferencing it stays infallible. pinned_for_restore verifies the profile before it compares the snapshot's digest with the constant, so a defective build does not pass for a mismatched snapshot. select_auto still builds every revision that covers the host, to select among them by ID and generation, but verifies only the one that it selects, so a Genoa host, which two revisions cover, verifies one. pinned_profiles, which only tools and tests call, panics on an invalid profile. Verification adds to the first use of a profile in each process, which NVX's single-process OpenVMM makes once per cold boot or restore. In fresh processes of a release build, the medians of the first select or pinned_for_restore rise by about 42 us on a Core i9-12900H, 55 us on a Xeon Platinum 8370C Azure VM, and 100 to 160 us on Xeon Silver 4114 (Skylake-SP) hosts at idle clocks; after a busy-wait that raises the clock, as the work before selection in a starting OpenVMM would, by 40 us on Linux and 100 to 180 us on Windows on those Skylake-SP hosts. Validation and the encoding take about a third and two thirds of it. Also checking the digest would have hashed the 9 KB encoding at every start, which nearly doubles the cost on CPUs without the SHA extensions, such as Skylake-SP. The crate documentation and the Guide's time and CPU compatibility section describe the verification. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Record the six commits that the first commit of this series pins as positions 362-367 of the microVM chain, and move the NVX pin to a417f946a. Positions 362-367 move NVX's time ABI to version 2: the shared host TSC bracket, the host OS's discipline of host UTC, the publication of host UTC paired with the guest TSC with the host's clock correction, the guest TSC's offset on every backend, the version-2 time sample and restore packet 5, and the Guide's description of them. They change the time-ABI backends, the portb device's time sample, the restore packet, and the microVM worker's time-abi unit, which positions 159-269 added, so they follow those positions upstream. Position 366 also adapts petri's microVM configuration, which PR15's position 088 adds, and the ttrpc microVM test, which PR25's position 146 adds, to the portb device's configuration and restore packet 5. Say so in the table's remarks. The pin is now 210 commits past the mapped tip. Positions 360 and 361 are no longer only a proposal: microsoft/openvmm#4807 merged them, so position 361, c32c57d12, is the head of feature/microvm/dev. Say so in the scope, and name microsoft/openvmm#4809 as the proposal of positions 362-367. Part of #1009 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions · 2026-10-29T20:22:03Z

2 Petri tests failed (3 unstable)

Record the six commits that the first commit of this series pins as positions 362-367 of the microVM chain, and move the NVX pin to a417f946a. Positions 362-367 move NVX's time ABI to version 2: the shared host TSC bracket, the host OS's discipline of host UTC, the publication of host UTC paired with the guest TSC with the host's clock correction, the guest TSC's offset on every backend, the version-2 time sample and restore packet 5, and the Guide's description of them. They change the time-ABI backends, the portb device's time sample, the restore packet, and the microVM worker's time-abi unit, which positions 159-269 added, so they follow those positions upstream. Position 366 also adapts petri's microVM configuration, which PR15's position 088 adds, and the ttrpc microVM test, which PR25's position 146 adds, to the portb device's configuration and restore packet 5. Say so in the table's remarks. The pin is now 210 commits past the mapped tip. Positions 360 and 361 are no longer only a proposal: microsoft/openvmm#4807 merged them, so position 361, c32c57d12, is the head of feature/microvm/dev. Say so in the scope, and name microsoft/openvmm#4809 as the proposal of positions 362-367. Part of #1009 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Record the four commits that the previous commit pins as positions 362-365 of the microVM chain, and move the NVX pin to e4c741e18. Positions 362-365 make the E_CPU_SURFACE errors of the microVM's CPU checks name every CPUID difference rather than the first 16, each entry of a snapshot's effective CPUID record on one line with the registers in which it differs, and name the non-zero CPUID entries outside the profile's tables that VP 0 reads together with the results that VP 0 does not observe. The VM worker then runs every CPU check of a partition, even after one fails, and reports their failures together. They change only the CPU checks of the VM worker's time ABI, which the time ABI work at positions 159-269 added, and the snapshot guide's description of them; say so in the table's remarks. The pin is now 208 commits past the mapped tip. Positions 360 and 361 are no longer only a proposal: microsoft/openvmm#4807 merged them, so position 361, c32c57d12, is the head of feature/microvm/dev. Say so in the scope, and name microsoft/openvmm#4804 as the proposal of positions 362-365. Refs #568 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Record the six commits that the first commit of this series pins as positions 362-367 of the microVM chain, and move the NVX pin to 75765b660. Positions 362-367 move NVX's time ABI to version 2: the shared host TSC bracket, the host OS's discipline of host UTC, the publication of host UTC paired with the guest TSC with the host's clock correction, the guest TSC's offset on every backend, the version-2 time sample and restore packet 5, and the Guide's description of them. They change the time-ABI backends, the portb device's time sample, the restore packet, and the microVM worker's time-abi unit, which positions 159-269 added, so they follow those positions upstream. Position 366 also adapts petri's microVM configuration, which PR15's position 088 adds, and the ttrpc microVM test, which PR25's position 146 adds, to the portb device's configuration and restore packet 5. Say so in the table's remarks. The pin is now 210 commits past the mapped tip. Positions 360 and 361 are no longer only a proposal: microsoft/openvmm#4807 merged them, so position 361, c32c57d12, is the head of feature/microvm/dev. Say so in the scope, and name microsoft/openvmm#4809 as the proposal of positions 362-367. Part of #1009 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

… roadmap Record the two commits that the previous commit pins as positions 368 and 369 of the microVM chain, and move the NVX pin to 6b409bd10. Position 368 makes the change of microsoft/openvmm#4803 unchanged: the KVP IC test boots Windows Server 2025 instead of 2022, whose COM activation race can fail the guest's answer to the test's IP information request. Position 369 stages the Server 2025 image for the check-in gates' x64 Linux KVM job. It excludes the test and its image from the MSHV job, where Server 2025's required initial reset fails. A rebase onto an upstream main that includes microsoft/openvmm#4803 drops position 368 and reconciles position 369 with upstream's changes for those jobs; say so in the table's remarks. The pin is now 212 commits past the mapped tip. Keep the re-checked chain in the scope: microsoft/openvmm#4807 merged positions 360 and 361, microsoft/openvmm#4804 merged positions 362-365, and microsoft/openvmm#4818 merged positions 366 and 367. Position 367, bcfc47cff, is therefore the head of feature/microvm/dev, and microsoft/openvmm#4816 proposes positions 368 and 369. Refs #1035 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Pin microsoft/openvmm#4807 at c32c57d129a36fc8c82308d6659f8d7ab5955474, two commits on bd2b27a2e, the previous pin, which is the head of feature/microvm/dev since microsoft/openvmm#4794 merged. At partition creation, for cold boot and restore, step 1 of the CPU profile verification in section 2.2.1.3 of the architecture guide requires the VMM to verify that the CPU profile is valid, including its canonical encoding, density, ownership masks, and CPU time bits. OpenVMM did so for host profiles, but built each pinned profile from the constants of pinned_data.rs on first use without validating or encoding it, so a stale or malformed pinned_data.rs reached partition creation unchecked; only the catalog's tests would have caught it. Its profile validation did not check the CPU time bits either, so the host profile document that a snapshot carries did not have to fix them on KVM and MSHV, whose preflights, unlike WHP's, do not check the time bits that the backend presents. The first commit adds the CPU time bits of section 3.2.3 to profile validation, naming every bit that violates them, so that decoding a host profile document that violates them, and so restoring its snapshot, fails with E_PROFILE_DIGEST. The second verifies each pinned profile once per process, on its first use: its built-in values must form a valid profile, CPU time bits included, whose canonical encoding is the built-in encoding that snapshots embed. A defective profile fails the cold boot or restore that selects it with E_PROFILE_DIGEST, before the partition exists. The verification encodes the profile but does not hash it: its digest stays a constant that OpenVMM's tests check against the golden digests. It adds about 40 to 55 us to the first profile use of each OpenVMM process, which NVX's single-process OpenVMM makes once per cold boot or restore, on current hosts, and 40 to 180 us on Skylake-SP hosts. The protocol vectors and built-in CPU profiles that nvx-rs copies are unchanged. Refs microsoft#567 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Record the two commits that the previous commit pins as positions 360 and 361 of the microVM chain, and move the NVX pin to c32c57d12. Positions 360 and 361 verify CPU profiles as the architecture guide's partition-creation checks require: profile validation checks the CPU time bits, so a snapshot's host profile document must fix them too, and the catalog verifies each pinned profile once per process, before a partition uses it, while the tests keep checking the golden digests. They change only cpu_profile, the fork's CPU profile crate that positions 159-269 add, a test helper of virt_mshv, and the snapshot guide, so they build on that fork-only code; say so in the table's remarks. The pin is now 204 commits past the mapped tip. Positions 355-359 are no longer only a proposal: microsoft/openvmm#4794 merged them, so position 359, bd2b27a2e, is the head of feature/microvm/dev. Say so in the scope, and name microsoft/openvmm#4807 as the proposal of positions 360 and 361. Refs microsoft#567 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

… roadmap Record the two commits that the previous commit pins as positions 370 and 371 of the microVM chain, and move the NVX pin to 549a21512. Position 370 makes the change of microsoft/openvmm#4803 unchanged: the KVP IC test boots Windows Server 2025 instead of 2022, whose COM activation race can fail the guest's answer to the test's IP information request. Position 371 stages the Server 2025 image for the check-in gates' x64 Linux KVM job. It excludes the test and its image from the MSHV job, where Server 2025's required initial reset fails. A rebase onto an upstream main that includes microsoft/openvmm#4803 drops position 370 and reconciles position 371 with upstream's changes for those jobs; say so in the table's remarks. The pin is now 214 commits past the mapped tip. Keep the re-checked chain in the scope: microsoft/openvmm#4807 merged positions 360 and 361, microsoft/openvmm#4804 merged positions 362-365, microsoft/openvmm#4818 merged positions 366 and 367, and microsoft/openvmm#4820 merged positions 368 and 369. Position 369, 5e0ab772e, is therefore the head of feature/microvm/dev, and microsoft/openvmm#4816 proposes positions 370 and 371. Refs #1035 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Pedro Henrique Penna (ppenna) and others added 2 commits October 29, 2026 11:01

Pedro Henrique Penna (ppenna) requested a review from a team as a code owner October 29, 2026 18:13

github-actions Bot added the Guide label Oct 29, 2026

Pedro Henrique Penna (ppenna) mentioned this pull request Oct 29, 2026

openvmm: verify pinned CPU profiles at partition creation microsoft/nvx#1016

Merged

Pedro Henrique Penna (ppenna) merged commit c32c57d into feature/microvm/dev Oct 29, 2026
74 checks passed

Pedro Henrique Penna (ppenna) mentioned this pull request Oct 29, 2026

[DO NOT REVIEW] microVM: publish the host's clock correction with each time sample #4809

Open

This was referenced Oct 29, 2026

[DO NOT REVIEW] microVM: name every CPUID difference in CPU-surface errors #4804

Merged

openvmm: name every CPUID difference in CPU-surface errors microsoft/nvx#1014

Merged

This was referenced Oct 30, 2026

[DO NOT REVIEW] vmm_tests: run the KVP IC test on Windows Server 2025 (port of #4803) #4816

Merged

openvmm: run the KVP IC test on Windows Server 2025 microsoft/nvx#1036

Merged

Navigation Menu

Repository navigation

[DO NOT REVIEW] cpu_profile: verify pinned CPU profiles at partition creation - #4807

[DO NOT REVIEW] cpu_profile: verify pinned CPU profiles at partition creation#4807
Pedro Henrique Penna (ppenna) merged 2 commits into
feature/microvm/devfrom
copilot/microvm-validate-pinned-cpu-profiles

Pedro Henrique Penna (ppenna) commented Oct 29, 2026 •

edited

Loading

Uh oh!

Uh oh!

github-actions Bot commented Oct 29, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

Pedro Henrique Penna (ppenna) commented Oct 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

CPU time bits (c9ac93a63)

Pinned profile verification (c32c57d12)

Cost

Tests

Validation

Uh oh!

Uh oh!

github-actions Bot commented Oct 29, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Pedro Henrique Penna (ppenna) commented Oct 29, 2026 •

edited

Loading

CPU time bits (`c9ac93a63`)

Pinned profile verification (`c32c57d12`)