Repository navigation
[DO NOT REVIEW] microVM: publish the host's clock correction with each time sample - #4830
Merged
Pedro Henrique Penna (ppenna) merged 6 commits intoOct 31, 2026
Conversation
The KVM backend pairs a host time sample with the host TSC by reading the TSC on both sides of the sample, and pairs a vCPU's TSC with the host TSC the same way. Time ABI 2 pairs every time sample's host UTC with the guest TSC on every backend, and MSHV and WHP need the same bracket to measure the guest TSC's offset from the host TSC, so the bracket belongs with the time ABI rather than with one backend. Move HostTscBracket, which reads the host TSC around an operation with LFENCE on both sides, and the conversion of its half width to nanoseconds, cycles_to_ns_ceil and pair_host_sample, to virt::time_abi::tsc. Their arithmetic is available on every host, and only the TSC read itself on an x86-64 host. The KVM backend uses them unchanged. The cycle conversion test moves with them, and a new test checks the bracket's midpoint and its refusal of a TSC that went backwards. Part of microsoft/nvx#1009. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Time ABI 2 publishes, with every time sample, how fast host UTC runs, so that the guest applies the host's clock correction instead of estimating it over minutes. The host already knows that correction: its time service steers host UTC by it. Add host_clock_discipline, which reads it from the host OS, whatever time service runs there, rather than from a hypervisor feature: - On Linux, adjtimex with no modes: the frequency correction plus the tick's deviation from its nominal 10 ms, the status (STA_CLOCKERR is unreliable, STA_UNSYNC free-running, anything else synchronized), and the maximum error that the time service sets. It also names the clock's basis from the current clocksource: the TSC at the kernel's kHz calibration, or the TSC at the hypervisor's rate through the Hyper-V TSC page. - On Windows, GetSystemTimeAdjustmentPrecise: its increment is the performance counter's frequency, and Windows counts a second of UTC for every adjustment of the counter's ticks, so UTC runs at the increment over the adjustment of the counter's rate, and a larger adjustment slows it. Over 20 s, UTC ran 0.45 ppm slow against the counter on a Windows host that reported an adjustment of 10,000,005 for an increment of 10,000,000, and 53 ppm slow on one whose time service moved its adjustment from 10,000,501 to 10,000,588. An adjustment that Windows does not apply is a free-running clock with no correction, and a zero one an unknown clock. Under Hyper-V the performance counter runs at the hypervisor's TSC rate. Add host_suspended_ns, how far the monotonic clock that counts suspended time has run ahead of one that does not (CLOCK_BOOTTIME minus CLOCK_MONOTONIC, and the interrupt time minus the unbiased interrupt time), from which a change reveals a suspension, and HostClockKind::follows_utc_rate, which says whether the anchor's monotonic clock runs at host UTC's corrected rate, as Linux's does and Windows' does not, so that a step of host UTC shows against it. The tests check the Windows adjustment's arithmetic, its direction with the readings of the host above, and its rounding, that each host reports a status and a plausible correction, and the clock kinds. Part of microsoft/nvx#1009. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
In time ABI 1 the guest pairs host UTC with its own clock: it reads its clock around the port write that latches a time sample, and accepts the pairing only within 1 ms at boot and restore, or 50 us for its discipline, so a port write that a loaded host serves late costs a sample, and three cost a boot or a restore. It also estimates over minutes how fast host UTC runs, although the host's time service knows. Add virt::time_abi::publish, which computes the host's fields of a time sample under time ABI 2: - Host UTC paired with the guest TSC on the host: the tightest of up to 16 pairings of a host time sample with the host TSC, plus the guest TSC's offset from the host TSC, and the pairing's bound, which adds the offset's uncertainty. GuestTscOffset holds that offset; it measures it from guest TSC reads between two host TSC reads and keeps the tightest, and says whether two measurements can be the same offset. - The correction: how fast host UTC runs against the guest TSC counted at the declared rate, minus one, in ppm scaled by 2^16, that is (1 + host correction) * F / F_d - 1. On a restore it includes the native rate's deviation from the declared one. - A bound on that correction's error: the rounding when the host's clock and the backend's native rate come from one calibration, the kernel's kHz rate for KVM and the hypervisor's rate for MSHV and WHP, one kHz of the native rate when they do not, and none when the host's clocksource may not follow the TSC. - The status and maximum error of host UTC, as its time service reports them. - A disruption counter, which changes when the host's suspended time changes, when host UTC moves against the host monotonic clock by 1 ms or more beyond what the correction explains (Linux corrects both alike; on Windows the correction may change between samples by up to 500 ppm), and when the guest TSC's offset changes. Add the host-rate-offset-ppm test hook, which publishes host UTC as running that many ppm faster than it does from the process's first sample on, with the correction to match, so that tests can check that a guest follows a host clock whose rate differs from the TSC's. The publisher applies utc-offset-ms too, which the portb device applied until now. The tests check the correction's arithmetic against exact values, the bounds, the pairing of a sample, the disruptions of Linux and Windows clocks, the hooks, and a real host sample's pairing. Part of microsoft/nvx#1009. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pairing a time sample's host UTC with the guest TSC on the host needs the guest TSC at an instant of the host TSC while the VPs run. The guest TSC is never scaled, so it is the host TSC plus an offset that holds until the VMM writes a guest TSC or stops partition time; what differs between backends is how to learn it. Add TimeAbiBackend::guest_tsc_offset, which core calls at every start of the VM, with every VP bound and stopped, after the TSC writes of a cold boot, a restore, or a reset, and before any VP runs: - KVM returns VP 0's KVM_VCPU_TSC_OFFSET, which preflight verifies is the unscaled offset, so it is exact. - MSHV and WHP read VP 0's TSC between two host TSC reads, up to eight times, and keep the tightest reading; it ends early within 2,000 cycles. Both freeze partition time until a partition starts (MSHV after preflight and a reset, WHP from creation and after a reset, and both when the partition unit stops), so the measurement resumes it first, as the partition's start would right after, and the offset holds once the VPs run. Resuming time that runs is harmless on both. Preflight also reports what each backend's native rate measures, which bounds the error of the published correction: KVM_GET_TSC_KHZ is the kernel's calibration of the TSC to the kHz, and MSHV's and WHP's processor clock frequency is the hypervisor's rate to the Hz. The MSHV hardware test checks that the measurement resumes partition time and agrees with a direct read of VP 0's TSC, and a new WHP hardware test checks that the offset holds while time runs: on a Windows 11 host, two measurements 500 ms apart differed by 87 cycles, within the 3,700 cycles of their uncertainty. Part of microsoft/nvx#1009. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Move every microVM to NVX time ABI 2 (microsoft/nvx#1009). In time ABI 1, the guest estimates how fast its TSC runs against host UTC: it pairs host UTC samples with its own clock, bracketing each port write with reads of that clock, and steers the kernel's phase-locked loop over minutes. The host already knows that rate, because its time service steers host UTC by it, and the guest's brackets depend on how fast the host serves the port write. Time ABI 2 has the VMM pair host UTC with the guest TSC and publish the host's correction, which the guest applies instead of estimating it. The time sample, which writing 0xa7 to the portb status port latches, becomes version 2, 48 bytes: the generation counter, host UTC and the guest TSC at the same instant, the bound of their pairing, the status of host UTC, the disruption counter, the correction against the TSC at the declared rate, and the bounds of the correction's error and of host UTC's error. The portb device takes these host fields from a platform resource, TimeSampleSourceKind, which the VM worker resolves to the time ABI's clock; the device keeps the generation, the test-hook flag, and the sample-delay-us hook, which delays the latch. The worker's clock pairs host UTC with the guest TSC through the offset that the time-abi state unit measures at every start, before any VP runs: the partition unit now depends on the time-abi unit, so the measurement follows the TSC writes of a cold boot, a restore's synchronized set, or a reset, and precedes the VPs' release on every start, including a resume after a failed capture. An offset that changes across a start is a disruption. Until the first measurement, or when the host cannot pair a sample, the window stays empty, and the guest rejects the sample. The restore packet becomes version 5, which drops host UTC: the guest takes a time sample for its repair, which pairs host UTC with its TSC on the host, so the packet no longer latches UTC at its first selection, and its header shrinks to 24 bytes. CPUID leaf 0x40000002 reports time ABI 2.0 in EBX (0x00020000), and snapshots record time ABI version 2. A snapshot of time ABI 1 fails restore with E_MANIFEST_TIME and must be recaptured: its guest decodes only version-1 samples, and a version-4 packet with UTC. The CPU time bits do not change, so every pinned CPU profile serves time ABI 2 as it is. The portb device no longer takes utc-offset-ms, which the publisher applies, so petri's microVM configuration drops it too, and the guest of the ttrpc microVM test reads restore packet 5: version 5, and 17 header bytes after the range count instead of 25. The tests encode and decode the version-2 sample and the version-5 packet byte for byte, check that the portb device latches the source's fields into its window and leaves it empty without a sample, check that the unit's start measures the offset that samples use and that a changed offset is a disruption, and check that restore rejects a version-1 time contract. Part of microsoft/nvx#1009. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The snapshots page described restore packet version 4, with host UTC latched at its first selection, and the 16-byte time sample of time ABI 1. Describe restore packet version 5, which carries no UTC, and the 48-byte sample of time ABI 2, field by field: host UTC paired with the guest TSC on the host, the pairing's bound, the status of host UTC, the disruption counter, the host's clock correction against the TSC at the declared rate, and the bounds of the correction's and of host UTC's errors. Say how OpenVMM pairs the guest TSC (KVM's offset, and MSHV's and WHP's offset measured from VP 0's TSC at every start) and where it reads the correction (adjtimex on Linux, GetSystemTimeAdjustmentPrecise on Windows), that the correction includes a restore's rate deviation, that the samples discipline only the guest's wall clock and are no paravirtual clock, and that a snapshot of time ABI 1 must be recaptured. The CLI and gRPC references name restore packet version 5. Part of microsoft/nvx#1009. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
This PR modifies files containing For more on why we check whole files, instead of just diffs, check out the Rustonomicon |
Pedro Henrique Penna (ppenna)
added a commit
to microsoft/nvx
that referenced
this pull request
Oct 31, 2026
Pin microsoft/openvmm#4830 at 858b2f93aa78d35df6fdd217322a7f4cfb57b080, six commits on 3d7512af653717aa9123595e993a56cb8587106a, the previous pin and the head of feature/microvm/dev. They are the commits of microsoft/openvmm#4809, which this pin replaces, rebased from 5e0ab772e; git range-diff keeps all six identical, and the rebased tree is the conflict-free three-way merge of 37ff2f9a5 with the new base. Under time ABI 1 the guest paired host UTC with its own clock: nvx-time read CLOCK_REALTIME around the port write that latches a time sample and accepted the pairing only within 1 ms at boot and restore, or 50 us for its discipline, so a port write that a loaded host served late cost a sample, and three cost a boot or a restore. It then estimated how fast host UTC runs against its TSC by steering the kernel's PLL over minutes, although the host's time service already knows that rate. Time ABI 2 has OpenVMM pair each sample's host UTC with the guest TSC on the host and publish with it the host's clock correction, which the host OS reports (adjtimex on Linux, GetSystemTimeAdjustmentPrecise on Windows), the bound of the correction's error, the status and maximum error of host UTC, and a disruption counter, which changes on a host suspension, a step of host UTC, or a change of the guest TSC's offset. The version-2 sample is 48 bytes. CPUID leaf 0x40000002 reports time ABI 2.0 (EBX 0x00020000). Restore packet 5 drops host UTC, so its header shrinks to 24 bytes, and the guest's repair takes a time sample instead. The guest moves with the pin, since a guest of one time ABI fails the other's conformance check: - nvx::contract: TIME_ABI_VERSION 2 and RESTORE_PACKET_VERSION 5. - Pairing: the guest reads its TSC before and after the selector write, and CLOCK_REALTIME between those reads, and accepts a sample only if its TSC lies within that bracket, widened by the published pairing bound and 1 us of skew. It compares host UTC with CLOCK_REALTIME at the sample's TSC, moving its CLOCK_REALTIME read there at the rate at which Linux counts it; the offset's uncertainty is the pairing bound, half the bracket, and 2,000 ppm of that move. - Rate: from the host's correction and the TSC rate, which the guest reads once from the Hyper-V TSC frequency MSR (0x40000022) and keeps in its time state, the guest computes the adjtimex frequency that makes CLOCK_REALTIME, which Linux counts at its kHz rounding of the TSC rate, run at host UTC's rate. It clamps that frequency to Linux's 500 ppm limit and reports whether it did. - Discipline: boot steps the clock to host UTC and holds that frequency with STA_FREQHOLD, so that the kernel's PLL corrects only the phase. Every poll holds the latest correction and slews the measured offset, or steps it from 128 ms on. A changed disruption counter returns the daemon to its fast cadence. Boot, restore, and polls take up to three samples within 100 us; a boot that gets none fails its check, and a restore fails its repair. The maximum and estimated errors that the guest sets include host UTC's maximum error only when the host reports a synchronized clock; the guest never marks its clock unsynchronized on the host's behalf. - Restore: the repair reads packet 5 and takes its time sample in the same port transaction, then steps the clock and holds the frequency. - State: the time state records the host's status, correction, and correction bound, host UTC's maximum error, the disruption counter, and whether the frequency was limited. nvx-time sample prints a sample's fields and the guest's pairing, and the test hooks packet, sample, pair, frequency, and correction take the new fields. Templates record time ABI version 2, so the manifest reader refuses a template of time ABI 1, which must be recaptured. The fixtures' manifests, which OpenVMM's generator regenerated at the pin, differ from the previous ones only in that version. The template-key vectors keep the contract versions that their test fixes. Comments, the nvx-rs README, and nvx-snapshot name the new versions. OpenVMM's protocol vectors and CPU profiles are unchanged at the pin. Part of #1009 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna)
added a commit
to microsoft/nvx
that referenced
this pull request
Oct 31, 2026
Record the six commits that the first commit of this series pins, which microsoft/openvmm#4830 proposes, as positions 373-378 of the microVM chain, and move the NVX pin to 858b2f93a. Positions 373-378 move NVX's time ABI to version 2: the shared host TSC bracket, the host OS's discipline of host UTC, the publication of host UTC paired with the guest TSC and its correction, the guest TSC's offset on every backend, the version-2 time sample and restore packet 5, and the Guide's description. They change the time-ABI backends, portb device, restore packet, and VM worker that positions 159-269 added, so they follow those positions upstream: replayed on upstream main, they conflict where those files are missing. Position 377 also adapts petri's microVM configuration from PR15 and the ttrpc microVM test from PR25. Position 378 names restore packet 5 in the CLI reference, in another part of it than the host-loopback rule that position 370 adds. The six commits are directly on 3d7512af6, position 372 and the current head of feature/microvm/dev since microsoft/openvmm#4816 merged positions 371 and 372, on position 370, which microsoft/openvmm#4823 merged. Update the scope, physical DAG, after-tip count and table, and remarks. The pin is 221 commits past the mapped tip. Part of #1009 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This was referenced Oct 31, 2026
Pedro Henrique Penna (ppenna)
added a commit
to microsoft/nvx
that referenced
this pull request
Oct 31, 2026
Pin microsoft/openvmm#4830 at 858b2f93aa78d35df6fdd217322a7f4cfb57b080, six commits on 3d7512af653717aa9123595e993a56cb8587106a, the previous pin and the head of feature/microvm/dev. They are the commits of microsoft/openvmm#4809, which this pin replaces, rebased from 5e0ab772e; git range-diff keeps all six identical, and the rebased tree is the conflict-free three-way merge of 37ff2f9a5 with the new base. Under time ABI 1 the guest paired host UTC with its own clock: nvx-time read CLOCK_REALTIME around the port write that latches a time sample and accepted the pairing only within 1 ms at boot and restore, or 50 us for its discipline, so a port write that a loaded host served late cost a sample, and three cost a boot or a restore. It then estimated how fast host UTC runs against its TSC by steering the kernel's PLL over minutes, although the host's time service already knows that rate. Time ABI 2 has OpenVMM pair each sample's host UTC with the guest TSC on the host and publish with it the host's clock correction, which the host OS reports (adjtimex on Linux, GetSystemTimeAdjustmentPrecise on Windows), the bound of the correction's error, the status and maximum error of host UTC, and a disruption counter, which changes on a host suspension, a step of host UTC, or a change of the guest TSC's offset. The version-2 sample is 48 bytes. CPUID leaf 0x40000002 reports time ABI 2.0 (EBX 0x00020000). Restore packet 5 drops host UTC, so its header shrinks to 24 bytes, and the guest's repair takes a time sample instead. The guest moves with the pin, since a guest of one time ABI fails the other's conformance check: - nvx::contract: TIME_ABI_VERSION 2 and RESTORE_PACKET_VERSION 5. - Pairing: the guest reads its TSC before and after the selector write, and CLOCK_REALTIME between those reads, and accepts a sample only if its TSC lies within that bracket, widened by the published pairing bound and 1 us of skew. It compares host UTC with CLOCK_REALTIME at the sample's TSC, moving its CLOCK_REALTIME read there at the rate at which Linux counts it; the offset's uncertainty is the pairing bound, half the bracket, and 2,000 ppm of that move. - Rate: from the host's correction and the TSC rate, which the guest reads once from the Hyper-V TSC frequency MSR (0x40000022) and keeps in its time state, the guest computes the adjtimex frequency that makes CLOCK_REALTIME, which Linux counts at its kHz rounding of the TSC rate, run at host UTC's rate. It clamps that frequency to Linux's 500 ppm limit and reports whether it did. - Discipline: boot steps the clock to host UTC and holds that frequency with STA_FREQHOLD, so that the kernel's PLL corrects only the phase. Every poll holds the latest correction and slews the measured offset, or steps it from 128 ms on. A changed disruption counter returns the daemon to its fast cadence. Boot, restore, and polls take up to three samples within 100 us; a boot that gets none fails its check, and a restore fails its repair. The maximum and estimated errors that the guest sets include host UTC's maximum error only when the host reports a synchronized clock; the guest never marks its clock unsynchronized on the host's behalf. - Restore: the repair reads packet 5 and takes its time sample in the same port transaction, then steps the clock and holds the frequency. - State: the time state records the host's status, correction, and correction bound, host UTC's maximum error, the disruption counter, and whether the frequency was limited. nvx-time sample prints a sample's fields and the guest's pairing, and the test hooks packet, sample, pair, frequency, and correction take the new fields. Templates record time ABI version 2, so the manifest reader refuses a template of time ABI 1, which must be recaptured. The fixtures' manifests, which OpenVMM's generator regenerated at the pin, differ from the previous ones only in that version. The template-key vectors keep the contract versions that their test fixes. Comments, the nvx-rs README, and nvx-snapshot name the new versions. OpenVMM's protocol vectors and CPU profiles are unchanged at the pin. Part of #1009 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna)
added a commit
to microsoft/nvx
that referenced
this pull request
Oct 31, 2026
Record the six commits that the first commit of this series pins, which microsoft/openvmm#4830 proposes, as positions 373-378 of the microVM chain, and move the NVX pin to 858b2f93a. Positions 373-378 move NVX's time ABI to version 2: the shared host TSC bracket, the host OS's discipline of host UTC, the publication of host UTC paired with the guest TSC and its correction, the guest TSC's offset on every backend, the version-2 time sample and restore packet 5, and the Guide's description. They change the time-ABI backends, portb device, restore packet, and VM worker that positions 159-269 added, so they follow those positions upstream: replayed on upstream main, they conflict where those files are missing. Position 377 also adapts petri's microVM configuration from PR15 and the ttrpc microVM test from PR25. Position 378 names restore packet 5 in the CLI reference, in another part of it than the host-loopback rule that position 370 adds. The six commits are directly on 3d7512af6, position 372 and the current head of feature/microvm/dev since microsoft/openvmm#4816 merged positions 371 and 372, on position 370, which microsoft/openvmm#4823 merged. Update the scope, physical DAG, after-tip count and table, and remarks. The pin is 221 commits past the mapped tip. Part of #1009 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna)
added a commit
to microsoft/nvx
that referenced
this pull request
Oct 31, 2026
Pin microsoft/openvmm#4830 at 858b2f93aa78d35df6fdd217322a7f4cfb57b080, six commits on 3d7512af653717aa9123595e993a56cb8587106a, the previous pin and the head of feature/microvm/dev. They are the commits of microsoft/openvmm#4809, which this pin replaces, rebased from 5e0ab772e; git range-diff keeps all six identical, and the rebased tree is the conflict-free three-way merge of 37ff2f9a5 with the new base. Under time ABI 1 the guest paired host UTC with its own clock: nvx-time read CLOCK_REALTIME around the port write that latches a time sample and accepted the pairing only within 1 ms at boot and restore, or 50 us for its discipline, so a port write that a loaded host served late cost a sample, and three cost a boot or a restore. It then estimated how fast host UTC runs against its TSC by steering the kernel's PLL over minutes, although the host's time service already knows that rate. Time ABI 2 has OpenVMM pair each sample's host UTC with the guest TSC on the host and publish with it the host's clock correction, which the host OS reports (adjtimex on Linux, GetSystemTimeAdjustmentPrecise on Windows), the bound of the correction's error, the status and maximum error of host UTC, and a disruption counter, which changes on a host suspension, a step of host UTC, or a change of the guest TSC's offset. The version-2 sample is 48 bytes. CPUID leaf 0x40000002 reports time ABI 2.0 (EBX 0x00020000). Restore packet 5 drops host UTC, so its header shrinks to 24 bytes, and the guest's repair takes a time sample instead. The guest moves with the pin, since a guest of one time ABI fails the other's conformance check: - nvx::contract: TIME_ABI_VERSION 2 and RESTORE_PACKET_VERSION 5. - Pairing: the guest reads its TSC before and after the selector write, and CLOCK_REALTIME between those reads, and accepts a sample only if its TSC lies within that bracket, widened by the published pairing bound and 1 us of skew. It compares host UTC with CLOCK_REALTIME at the sample's TSC, moving its CLOCK_REALTIME read there at the rate at which Linux counts it; the offset's uncertainty is the pairing bound, half the bracket, and 2,000 ppm of that move. - Rate: from the host's correction and the TSC rate, which the guest reads once from the Hyper-V TSC frequency MSR (0x40000022) and keeps in its time state, the guest computes the adjtimex frequency that makes CLOCK_REALTIME, which Linux counts at its kHz rounding of the TSC rate, run at host UTC's rate. It clamps that frequency to Linux's 500 ppm limit and reports whether it did. - Discipline: boot steps the clock to host UTC and holds that frequency with STA_FREQHOLD, so that the kernel's PLL corrects only the phase. Every poll holds the latest correction and slews the measured offset, or steps it from 128 ms on. A changed disruption counter returns the daemon to its fast cadence. Boot, restore, and polls take up to three samples within 100 us; a boot that gets none fails its check, and a restore fails its repair. The maximum and estimated errors that the guest sets include host UTC's maximum error only when the host reports a synchronized clock; the guest never marks its clock unsynchronized on the host's behalf. - Restore: the repair reads packet 5 and takes its time sample in the same port transaction, then steps the clock and holds the frequency. - State: the time state records the host's status, correction, and correction bound, host UTC's maximum error, the disruption counter, and whether the frequency was limited. nvx-time sample prints a sample's fields and the guest's pairing, and the test hooks packet, sample, pair, frequency, and correction take the new fields. Templates record time ABI version 2, so the manifest reader refuses a template of time ABI 1, which must be recaptured. The fixtures' manifests, which OpenVMM's generator regenerated at the pin, differ from the previous ones only in that version. The template-key vectors keep the contract versions that their test fixes. Comments, the nvx-rs README, and nvx-snapshot name the new versions. SOURCE-MANIFEST.json declares the time ABI and restore packet versions too, and nvx-rs requires them wherever it checks a release's contracts: the bundled feature's build script and the artifact discovery refuse a release that lacks them, and nvx-rs download warns about it. Each names every contract field that differs, with the value that it expects and the one that the release declares, through contract::manifest_mismatches: a release from before time ABI 2 declares the current control contract revision, so naming the revision alone would point at a value that already matches. The distribution guide and the download command's usage say so. OpenVMM's protocol vectors and CPU profiles are unchanged at the pin. Part of #1009 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna)
added a commit
to microsoft/nvx
that referenced
this pull request
Oct 31, 2026
Record the six commits that the first commit of this series pins, which microsoft/openvmm#4830 proposes, as positions 373-378 of the microVM chain, and move the NVX pin to 858b2f93a. Positions 373-378 move NVX's time ABI to version 2: the shared host TSC bracket, the host OS's discipline of host UTC, the publication of host UTC paired with the guest TSC and its correction, the guest TSC's offset on every backend, the version-2 time sample and restore packet 5, and the Guide's description. They change the time-ABI backends, portb device, restore packet, and VM worker that positions 159-269 added, so they follow those positions upstream: replayed on upstream main, they conflict where those files are missing. Position 377 also adapts petri's microVM configuration from PR15 and the ttrpc microVM test from PR25. Position 378 names restore packet 5 in the CLI reference, in another part of it than the host-loopback rule that position 370 adds. The six commits are directly on 3d7512af6, position 372 and the current head of feature/microvm/dev since microsoft/openvmm#4816 merged positions 371 and 372, on position 370, which microsoft/openvmm#4823 merged. Update the scope, physical DAG, after-tip count and table, and remarks. The pin is 221 commits past the mapped tip. Part of #1009 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna)
merged commit Oct 31, 2026
858b2f9
into
feature/microvm/dev
205 of 210 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Moves every microVM to NVX time ABI 2, for microsoft/nvx#1009: OpenVMM pairs each time sample's host UTC with the guest TSC on the host and publishes with it the host's clock correction, which the guest applies instead of estimating over minutes how fast host UTC runs.
Replaces #4809, rebased onto
feature/microvm/dev. This branch is #4809's six commits rebased from5e0ab772eonto3d7512af6, the head offeature/microvm/devsince #4816 merged. #4809 stays open and unchanged at37ff2f9a5.git range-diffkeeps all six commits identical, and the rebased tree is exactly the conflict-free three-way merge of37ff2f9a5with the new base (see Validation). The second commit retains the earlier fix that reads Windows' precise time adjustment in the right direction. microsoft/nvx#1061, a draft, pins this head.Why
Under time ABI 1 the guest pairs host UTC with its own clock: it reads that clock around the port write that latches a time sample and accepts the pairing only within 1 ms at boot and restore, or 50 µs for its discipline. A port write that a loaded host serves late costs a sample, and three cost a boot or a restore. The guest then estimates how fast host UTC runs against its TSC by steering the kernel's phase-locked loop over minutes, although the host's time service already knows the correction that it applies to host UTC. Time ABI 2 has the VMM pair host UTC with the guest TSC, which does not depend on how fast the host serves the port write, and publish the host's correction.
Shared host TSC bracket (
a34095ab3)HostTscBracket, which reads the host TSC around an operation with LFENCE on both sides,cycles_to_ns_ceil, andpair_host_samplemove from the KVM backend tovirt::time_abi::tsc, since every backend now needs them. The KVM backend uses them unchanged.Host UTC's discipline (
26ebbe98c)host_clock_disciplinereads the host's correction from the host OS, whatever time service runs there, rather than from a hypervisor feature:adjtimexwith no modes: the frequency correction plus the tick's deviation from its nominal 10 ms, the status (STA_CLOCKERRis unreliable,STA_UNSYNCfree-running, anything else synchronized), and the maximum error. The current clocksource names the clock's basis: the TSC at the kernel's kHz calibration, or at the hypervisor's rate through the Hyper-V TSC page.GetSystemTimeAdjustmentPrecise. Its increment is the performance counter's frequency, and Windows counts a second of UTC for every adjustment's worth of the counter's ticks. UTC therefore runs at the increment over the adjustment of the counter's rate, and a larger adjustment slows it. An adjustment that Windows does not apply is a free-running clock, and a zero adjustment an unknown one.host_suspended_nsreveals a host suspension:CLOCK_BOOTTIMEminusCLOCK_MONOTONIC, and the interrupt time minus the unbiased interrupt time.Publishing (
d5857a5cc)virt::time_abi::publishcomputes the host's fields of a time sample:(1 + host correction) * F / F_d - 1. On a restore it includes the native rate's deviation from the declared one.It adds the
host-rate-offset-ppmtest hook, which publishes host UTC as running that many ppm faster than it does, with the correction to match, so that tests can check that a guest follows a host clock whose rate differs from the TSC's. The publisher also appliesutc-offset-ms, which the portb device applied until now.The guest TSC's offset on every backend (
06e7daf69)TimeAbiBackend::guest_tsc_offsetruns at every start of the VM, with every VP bound and stopped, after the TSC writes of a cold boot, a restore, or a reset, and before any VP runs:KVM_VCPU_TSC_OFFSET, which preflight verifies is the unscaled offset.Preflight also reports what each backend's native rate measures, which bounds the published correction's error.
Time ABI 2 (
69e1c9826)The time sample that writing
0xa7to the portb status port latches becomes version 2, 48 bytes, little-endian:0xffffffffunknown0xffffffffunknownThe portb device takes the host's fields from a platform resource,
TimeSampleSourceKind, which the VM worker resolves to the time ABI's clock; the device keeps the generation, the test-hook flag, and thesample-delay-ushook. The partition unit now depends on the time-abi unit, so the offset's measurement follows the TSC writes of every start and precedes the VPs' release, including a resume after a failed capture. Until the first measurement, or when the host cannot pair a sample, the window stays empty and the guest rejects the sample.The restore packet becomes version 5, which drops host UTC: the guest takes a time sample for its repair, so the packet no longer latches UTC at its first selection, and its header shrinks to 24 bytes.
CPUID leaf
0x40000002reports time ABI 2.0 in EBX (0x00020000), and snapshots record time ABI version 2. A snapshot of time ABI 1 fails restore withE_MANIFEST_TIMEand must be recaptured. The CPU time bits do not change, so every pinned CPU profile serves time ABI 2 as it is.Petri's microVM configuration no longer sets the portb device's
utc_offset_ms, which the publisher applies now, and the guest of the ttrpc microVM VMM test reads restore packet 5: version 5, and 17 header bytes after the range count instead of 25.Guide (
858b2f93a)The snapshots page describes restore packet 5 and the version-2 sample field by field, how OpenVMM pairs the guest TSC and where it reads the correction, that the samples discipline only the guest's wall clock and are no paravirtual clock, and that a snapshot of time ABI 1 must be recaptured. The CLI and gRPC references name restore packet version 5.
Tests
virt::time_abi::tsc: the cycle conversion, and the bracket's midpoint and its refusal of a TSC that went backwards.virt::time_abi::host: the Windows adjustment's arithmetic, its direction with two Windows hosts' readings, and its rounding, that each host reports a status and a plausible correction, and the clock kinds.virt::time_abi::publish: the correction's arithmetic against exact values, its bounds, the pairing of a sample, the disruptions of Linux and Windows clocks, the hooks, and a real host sample's pairing.chipset_resources::microvm_time: the version-2 sample and the version-5 packet, encoded and decoded byte for byte.chipset::microvm: the portb device latches the source's fields into its window, and leaves it empty without a sample.openvmm_core: the time-abi unit's start measures the offset that samples use, and a changed offset is a disruption. A restore rejects a version-1 time contract.Validation
This rebase, at
858b2f93afeature/microvm/devmoved from5e0ab772eto3d7512af6when [DO NOT REVIEW] microVM: reject explicit host-loopback access on a /30 IPv4 network #4823 (explicit host-loopback access refused on a /30 IPv4 network) and [DO NOT REVIEW] vmm_tests: run the KVP IC test on Windows Server 2025 (port of #4803) #4816 (the KVP IC VMM test on Windows Server 2025, and the check-in gates' KVM and MSHV jobs) merged.git rebaseapplied all six commits without conflicts.git range-diff 5e0ab772e..37ff2f9a5 3d7512af6..858b2f93amarks all six identical (=), messages included; their complete diffs against their bases match line for line apart from hunk offsets in the CLI reference; and the rebased tree,5d3b9c03e, is exactlygit merge-tree --merge-base 5e0ab772e 3d7512af6 37ff2f9a5. The three new base commits share only the CLI reference with this branch, in another section (--host-loopbackrather than--restore-entropy), and touch none of the time ABI's code.virt,chipset,chipset_resources,openvmm_core,openvmm_defs,openvmm_entry,openvmm_helpers,petri) withvirt_kvmandvmm_testspasscargo checkandcargo clippy --all-targets -- -D warnings, and withvirt_kvmrustdoc; 732 unit tests and 13 ignored hardware tests (virt_kvm's 12 andvirt's host identity) pass; andcargo xflowey vmm-tests-runpasses both microVM VMM tests,ttrpc::test_ttrpc_microvm_linux_direct_lifecycle_and_snapshotandx86_64::microvm::openvmm_linux_x64_phase_1_lifecycle.virt_mshvpass check, clippy, and rustdoc; 770 unit tests and 14 hardware tests pass; and CI's quick-check clippy,cargo clippy --locked --keep-going --all-targets --workspace --all-features --target x86_64-unknown-linux-gnu --exclude guest_test_uefi --exclude opentmk_executor -- -Dwarnings, passes.virt_whppass check, clippy, and rustdoc; 698 unit tests and 19 hardware tests (virt_whp's 18 andvirt's host identity) pass.cargo xtask fmtpasses: rustfmt, the house lints, andverify-flowey, which checks the pipelines that [DO NOT REVIEW] vmm_tests: run the KVP IC test on Windows Server 2025 (port of #4803) #4816 regenerated.dev. With this OpenVMM, NVX's default microVM suite (both shards for each backend, 42 scenarios,time-host-correctionamong them) passes on the KVM, MSHV, and WHP test VMs, as donvx-rs's comparisons of its copies of the control-session vectors, source excerpts, contract versions, and CPU profiles with this revision, and the generator writes NVX's template fixtures byte for byte.858b2f93a: in progress, https://github.com/microsoft/openvmm/actions/runs/41149160729.Earlier heads, as #4809
cargo clippy --all-targets -- -D warnings,cargo doc --no-deps, andcargo nextest runfor the changed packages, the WHP time-ABI hardware tests, andcargo xtask fmt: clean.cargo nextest runfor the changed packages (796 passed) and the KVM time-ABI hardware tests (5 passed).cargo doc --no-deps.cargo clippy --all-targets -- -D warningsfor every changed package found twouseless_conversionerrors inhost.rs, which the second commit now fixes; clippy andvirt'stime_abitests (66) then passed.utc_offset_ms. The fifth commit now drops it, and the quick check's clippy,cargo clippy --locked --keep-going --all-targets --workspace --all-features --target x86_64-unknown-linux-gnu --exclude guest_test_uefi --exclude opentmk_executorwith-Dwarnings, passes on the Azure MSHV VM. Looking for other users of the changed formats, the ttrpc microVM VMM test's guest still parsed restore packet 4, so the fifth commit updates it too.cargo xflowey vmm-tests-runpassed both microVM VMM tests,ttrpc::test_ttrpc_microvm_linux_direct_lifecycle_and_snapshotandx86_64::microvm::openvmm_linux_x64_phase_1_lifecycle, on the Azure KVM VM, and the ttrpc one failed without the update (RESTORE-PACKET-INVALID).bcfc47cff, at090fbab88:git range-diffkeeps commits 1-3 and 5-6 patch-equivalent and adds only the newer base'sCpuBackend::guest_tsc_offsetmock to commit 4. On Windows, package-scoped check, clippy with-D warnings, rustdoc, and 697 unit tests pass for every generic and WHP crate; 18 ignored WHP hardware tests andcargo xtask fmt --fixpass. On Linux/KVM, the generic and KVM crates pass check, clippy, rustdoc, 731 unit tests, 12 ignored hardware tests, and both affected VMM tests. On Linux/MSHV,virt_mshvpasses check, clippy, rustdoc, 79 unit tests, and 13 ignored hardware tests.5e0ab772e, at37ff2f9a5:git range-diffkeeps all six commits patch-equivalent. [DO NOT REVIEW] vmcore: map VM times beyond the host clock to its latest time #4820 changes onlyvmcore's mapping of far VM times to host monotonic time; time ABI 2 samples host UTC and the TSC directly and reads only the VM-time stop sample on capture, so its sampling, disruption, and restore paths need no adjustment. On Windows, the packages pass check, clippy, rustdoc, 730 unit tests, formatting, and 18 WHP hardware tests. Linux/KVM passes those package gates with 764 unit tests, 12 hardware tests, and both affected VMM tests; Linux/MSHV passes with 194 unit tests and 13 hardware tests.adjtimexreports synchronized clocks on the Azure Linux test VMs and all ten Linux CI runners, at corrections from -76 to +7 ppm, andGetSystemTimeAdjustmentPrecisereports applied adjustments on an Azure Windows VM, on all ten Windows CI runners, from -72.5 to +6.9 ppm, and at +17.2 ppm on a bare-metal Windows host (read in the right direction; see below).time-host-correction, which checks with thehost-rate-offset-ppmhook that the guest follows a host clock that runs at another rate than the TSC. With this branch's OpenVMM, NVX's default microVM suite and the scenario passed on Linux/KVM, Linux/MSHV, and Windows/WHP Azure test VMs.GetSystemTimeAdjustmentPreciseas the legacyGetSystemTimeAdjustment, which adds its adjustment once per increment.QueryPerformanceCountershowed that a larger adjustment slows UTC. On that runner, whose adjustment moved from 10,000,501 to 10,000,588 for an increment of 10,000,000, UTC ran 53.1 ppm slow. On the Windows test VM, at an adjustment of 10,000,005, UTC ran 0.447 ppm slow.virt's formatting, clippy with-Dwarnings, and its time ABI tests (66) pass on Windows and Linux.75765b660passed. CI for090fbab88passed on attempt 10: https://github.com/microsoft/openvmm/actions/runs/41015566000. Attempts 1-9 failed only the flakyx64-windows-amd-snpVMM-test matrix, in varying unrelated Hyper-V/OpenHCL tests; attempt 10 passed all 98 tests. Exact-head CI for37ff2f9a5passed on attempt 2: https://github.com/microsoft/openvmm/actions/runs/41099502164. Attempt 1's AMD and TDX VMM jobs failed before testing when artifact downloads exhausted five retries; rerunning those jobs passed. The earlier binary-size check needed its base commit's CI run as a baseline and passed on rerun; the size comparison found nothing. The Petri tests that the bot reports are OpenHCL UEFI Windows tests marked unstable, which timed out.Notes
37ff2f9a5; close it once this lands.feature/microvm/devto858b2f93aa78d35df6fdd217322a7f4cfb57b080.