Repository navigation
openvmm: map VM times beyond the host clock to its latest time - #1042
Open
Pedro Henrique Penna (ppenna) wants to merge 2 commits into
Open
Pedro Henrique Penna (ppenna) wants to merge 2 commits into
Pedro Henrique Penna (ppenna) wants to merge 2 commits into
Conversation
Pin microsoft/openvmm#4820 at 5e0ab772eeea14f4d0f5ed05474856569358f80e, two commits on bcfc47cff, the previous pin, which is the head of feature/microvm/dev. OpenVMM's VM time keeper maps a VM time to the host time at which VM time reaches it, to arm its host timer and for VmTimeAccess::host_time, from the time's circular distance after the last start. A timeout more than about 584 years ahead, beyond what the host clock's 64 bits of nanoseconds can represent, overflowed that host time and panicked the task that armed it: a timeout armed across a restore to a VM time that it follows by 2.66e18 units panicked the keeper's task at start(). A timeout nearly half the ring, about 29,000 years, ahead of now, which VM time's circular order puts before the start once VM time has run, mapped to the start's host time, in the past, so the keeper's host timer fired and was re-armed forever, with the timer state's lock held. The first commit saturates the host time at the host clock's latest time, a deadline that every timer backend accepts and never reaches. The second maps a time that the circular order puts before the start, but after now, to that latest time too. Each adds tests that fail without it. The protocol vectors and built-in CPU profiles that nvx-rs copies are unchanged. Fixes #1038 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
The roadmap’s reference to “the two commits” is ambiguous and should explicitly identify positions 368 and 369 as independently upstreamable.
1 open issue
What changed in this PR
Updates NVX’s OpenVMM pin to address extreme VM deadlines that could panic the time-keeper task or make it spin, and documents the fixes in the upstream roadmap.
Changes:
- Pins OpenVMM to
5e0ab772e, which maps unrepresentable deadlines to the host clock’s latest time. - Records positions 368–369, refreshes the pin’s lineage and commit counts, and explains the fixes’ independent upstream applicability.
| File | Description |
|---|---|
doc/openvmm-upstream-roadmap.md |
Updates the pinned revision, chain mapping, merge history, and VM-time fix descriptions. |
Review effort: Max
Record the two commits that the previous commit pins as positions 368 and 369 of the microVM chain, and move the NVX pin to 5e0ab772e. Positions 368 and 369 make OpenVMM's VM time keeper map a timeout beyond what the host clock can represent to the host clock's latest time: one more than about 584 years after the last start no longer panics the task that arms it, and one nearly half the ring after now, which VM time's circular order puts before the start, no longer arms the host timer in the past. They change only vmcore's vmtime.rs, which upstream main and the fork base have alike, and apply to upstream main as they are; say so in the table's remarks. The pin is now 212 commits past the mapped tip. Position 367, bcfc47cff, is the head of feature/microvm/dev since microsoft/openvmm#4818 merged positions 366 and 367. Say so in the scope, and name microsoft/openvmm#4820 as the proposal of positions 368 and 369. Refs #1038 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna)
force-pushed
the
agents/issue-1038-debug-test-fix-commit
branch
from
October 30, 2026 06:27
bae0fe4 to
6d3442d
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Fixes #1038. Pins microsoft/openvmm#4820 at
5e0ab772eeea14f4d0f5ed05474856569358f80e, which makes OpenVMM's VM time keeper map a timeout beyond what the host clock can represent to the host clock's latest time. Such a timeout panicked the task that armed it, as the issue's sequence of public calls panics the keeper's task atstart(), and one nearly half the ring ahead of now made the keeper spin. This pull request also maps the pin in the OpenVMM roadmap.Based on
devat4434b1af0. microsoft/openvmm#4820 is two commits directly onbcfc47cff, the head offeature/microvm/devand the revision thatdevpins since #1039 merged, and the pin is that pull request's head commit itself, so this branch's two commits are its whole diff.Change
70adf8918openvmmopenvmmbcfc47cff→5e0ab772e: the two commits of microsoft/openvmm#48206d3442d14docdoc/openvmm-upstream-roadmap.mdvmcore's VM time keeper, whosevmtime.rsupstream main and the fork base have alike, and apply to upstream main as they are; moves the NVX pin; and records that microsoft/openvmm#4818 merged positions 366 and 367 intofeature/microvm/devNo other NVX change is needed. OpenVMM's control-session protocol vectors and built-in CPU profiles are unchanged at the pin.
OpenVMM changes
VmTimecounts 100 ns units in 64 bits and orders them circularly: a time less than half the ring, about 29,000 years, after another follows it.TimerState::timestampmaps a VM time to the host time at which VM time reaches it, from its circular distance after the last start.set_nextarms the keeper's host timer with it, andVmTimeAccess::host_timereturns it.342887c99). pal_async'sInstantcounts nanoseconds in 64 bits, about 584 years, and itsAddpanics on overflow, so a timeout more than about 584 years after the start panicked the task that armed it. In the issue's sequence, a timeout armed while VM time is stopped, at 1.8e16, follows the restored time, 1.58e19, by 2.66e18 units, sostart()panics the time source's task inTimerState::start.timestampnow adds the distance withInstant::saturating_add, so such a time maps to the host clock's latest time, which the host clock never reaches. Every timer backend already accepts that deadline: a timeout just short of overflowing reached it before, and the epoll, kqueue, IOCP, thread pool, io_uring, and local backends saturate their own conversions of it.5e0ab772e). Once VM time has run since the start, a time nearly half the ring after now is more than half the ring after the start, which the circular order puts before it, sotimestampmapped it to the start's host time, in the past. The waiters compare with now and keep waiting, so the host timer fired at once,wakere-armed the same deadline, andTimerState::polllooped forever with the timer state's write lock held.timestampnow maps a time that the circular order puts before the start, but after the current time, to the host clock's latest time too. It reads the current time only in that case.A saved state can carry any VM time, as in the issue. A guest can choose both kinds of timeout where OpenVMM emulates the Hyper-V synthetic timers, as WHP does when it does not offload the enlightenments: a one-shot timer expires at the absolute reference time that the guest writes, up to half the ring after the current one.
The issue suggests saturating the distance at a day instead, after
TimerState::poll's default deadline. That stops the panic but makes the keeper spin once VM time has run for a day since the start, since every clamped deadline is then in the past: aTimerStatestarted two days ago with a timeout 1,000 years ahead never returns frompollwith that clamp, and returns at once with the saturation.Validation
nvx-kvm-test(Azure Linux 3), at each commit:cargo nextest run --profile agent -p vmcore(32 and 33 tests passed) andcargo doc --no-deps -p vmcore. On Windows, alsocargo clippy --all-targets -p vmcore -- -D warningsandcargo xtask fmt --fix, which the test VM's toolchain lacks. All clean.bcfc47cff, the previous pin, on both: the issue's sequence panics with "duration too large" inInstant::add, with the issue's backtrace (TimerState::timestamp←set_next←wake←TimerState::start←SecondaryKeeper::run); a timeout 1,000 years ahead of a running keeper panics inhost_time; and a timeout nearly half the ring ahead makes the keeper spend 9.99 s of user CPU time in 10 s, onnvx-kvm-test, until it is killed. Each new test fails without its commit.nvx-kvm-test(Linux/KVM) andnvx-mshv-test(Linux/MSHV), with OpenVMM built at the pin, whose provenance records5e0ab772eand a clean tree, and the guest artifacts of releasev0.1.0-dev.d34d8ab730e5:python3 scripts/nvx.py test-microvmwith the default suite, 41 scenarios on the Alpine guest with 1, 2, 4, and 8 vCPUs, among themrestore-downtime,snapshot-core,smp-snapshot,console-snapshot, andtime-abi-conformance, passed on both backends, all 47 scenario records of each.6d3442d14:python scripts\nvx.py verify(consistent); theopenvmm_vectorsandopenvmm_cpu_profilescomparisons with--ignored(3 and 1 passed) against the pinned revision;python scripts\check_markdown_links.py; andgit diff --check.6d3442d14, every check passed (run 41014162393), the microVM tests on Linux/KVM, Linux/MSHV, and Windows/WHP among them. The first head,bae0fe48e, differed only in the roadmap remark that the review clarified, and its run was superseded. [DO NOT REVIEW] vmcore: map VM times beyond the host clock to its latest time openvmm#4820's checks all passed at5e0ab772e: the first attempt of its run 41011644930 failed two Windows vmm-tests jobs on failures outside this change, which their rerun passed. Two SNP tests panicked inopenhcl_boot, which does not depend onvmcore, with "Imported regions hash mismatch", as that job did in at least seven other pull requests' runs of the past day, andhibernate_tokenfailed withos error 10022in petri's harness before OpenVMM started.nvx-whp-test's SSH host key changed, so it was not used, and the bare-metal WHP host was running another OpenVMM.Merge microsoft/openvmm#4820 into
feature/microvm/dev, by fast-forwarding that branch to5e0ab772e, before merging this pull request.