[Pedro Henrique Penna (ppenna)]

Fixes the VM time keeper's panic of microsoft/nvx#1038, and a spin of the keeper next to it. A timeout that VM time reaches beyond what the host clock can represent, more than about 584 years ahead, now maps to the host clock's latest time instead of panicking the task that arms it. So does a timeout nearly half the ring, about 29,000 years, ahead of now, which mapped to a host time in the past and made the keeper spin.

Based on feature/microvm/dev. This branch is two commits directly on bcfc47cff, the head of feature/microvm/dev since #4818 merged, which microsoft/nvx's dev pins. microsoft/nvx#1042 pins this branch's head.

Why

VmTime counts 100 ns units in 64 bits and orders them circularly: a time less than half the ring after another follows it. TimerState::timestamp maps a VM time to the host time at which VM time reaches it, by adding the time's circular distance from the last start to the host time of that start. set_next arms the keeper's host timer with it, and VmTimeAccess::host_time returns it. Two kinds of time broke it:

• Beyond the host clock. pal_async's Instant counts nanoseconds in 64 bits, about 584 years, and its Add panics on overflow, so a timeout more than about 584 years after the start panicked the task that armed it. In vmtime: armed timeout + restore to a large time panics the keeper task on start() nvx#1038's sequence, a timeout is armed while VM time is stopped, and VM time is restored to a time that the timeout follows by 2.66e18 units, so start() panics the time source's task with "duration too large".
• Nearly half the ring ahead. Once VM time has run since the start, a time nearly half the ring after now is more than half the ring after the start, which the circular order puts before it, so timestamp mapped it to the host time of the start, in the past. The waiters compare with now, though, and keep waiting: the host timer fired at once, wake re-armed the same deadline, and TimerState::poll looped forever with the timer state's write lock held.

A guest can choose both kinds where OpenVMM emulates the Hyper-V synthetic timers, as WHP does when it does not offload the enlightenments: a one-shot timer expires at the absolute reference time that the guest writes, up to half the ring after the current one.

The issue suggests clamping the distance at a day, after TimerState::poll's default deadline. That stops the panic but makes the keeper spin, as above, once VM time has run for a day since the start, since the clamped deadline is then in the past: a TimerState started two days ago with a timeout 1,000 years ahead never returns from poll with it.

Saturation (342887c99)

timestamp adds the distance with Instant::saturating_add, so a time beyond the host clock maps to its latest time, which the host clock never reaches. Every timer backend already accepts that deadline: a timeout just short of overflowing reached it before, and the epoll, kqueue, IOCP, thread pool, io_uring, and local backends saturate their own conversions of it.

test_timeout_beyond_host_clock arms a timeout 1,000 years ahead of a running keeper and checks that host_time returns the host clock's latest time. test_timeout_beyond_host_clock_after_restore follows the issue's sequence. Both check that the timeout keeps waiting and that an earlier timeout still fires, and both panic with "duration too large" without the fix.

Times nearly half the ring ahead (5e0ab772e)

timestamp maps a time that the circular order puts before the start, but after the current time, to the host clock's latest time too. It reads the current time only for a time before the start, so the common case costs nothing more. A time before the start and not after now is in the past, and still maps to the host time of the start.

test_timeout_nearly_half_the_ring_ahead sets a timeout at the latest time that follows now, after VM time has run for 10 ms, and checks that a time in the past before the start still maps to the start. Without the fix, host_time returns the host time of the start, and without that check, the wait spins until nextest kills the test.

Validation

• On Windows and on Linux (Azure Linux 3), at each commit: cargo nextest run --profile agent -p vmcore (32 and 33 tests passed) and cargo doc --no-deps -p vmcore. On Windows, also cargo clippy --all-targets -p vmcore -- -D warnings and cargo xtask fmt --fix. All clean.
• At bcfc47cff, on both, the new tests fail as described: the two overflow tests panic in Instant::add, the issue's with its backtrace (timestamp ← set_next ← wake ← TimerState::start ← SecondaryKeeper::run), and the wait for a timeout nearly half the ring ahead spends 9.99 s of user CPU time in 10 s until it is killed.
• End to end, with this branch pinned by openvmm: map VM times beyond the host clock to its latest time nvx#1042, NVX's default microVM suite, 41 scenarios with 1, 2, 4, and 8 vCPUs, passed on Linux/KVM and Linux/MSHV test VMs, and openvmm: map VM times beyond the host clock to its latest time nvx#1042's CI passed.
• CI: run 41011644930 passed on its second attempt. Its first failed two Windows vmm-tests jobs on failures outside this change, which their rerun passed. In x64-windows-amd-snp, two SNP tests panicked in openhcl_boot, which does not depend on vmcore, with "Imported regions hash mismatch", as that job did in at least seven other pull requests' runs of the past day. In x64-windows-intel, hibernate_token failed with os error 10022 in petri's harness before OpenVMM started.
• Every commit is signed.

[github-actions]

4 Petri tests failed (2 unstable)