Repository navigation
scripts: keep the guest console's echo out of microVM test markers - #991
Merged
Pedro Henrique Penna (ppenna) merged 4 commits intoOct 29, 2026
Merged
Conversation
test-microvm typed each guest script into the guest's interactive shell line by line. The guest's terminal echoes console input that arrives while a command runs into whatever that command prints, so when a slow guest received a script in pieces, the echo of its later lines could land inside the markers that its earlier lines printed (#785). In CI run 39902391325, the ipv6-egress-policy case allow-ipv6-tcp-range read 'none0000 &' as its DNS server: the tail of the UDP probe's line had landed before the newline of "NVX-IPV6-EGRESS-DNS none". The scenarios that boot through nvx-rs run send their script as soon as the boot marker appears, without the time ABI status query that the other cold boots wait for, so their input can even reach the guest before its shell starts. run_guest_script now stores the script in the guest with a here-document, which the shell reads whole before it runs anything. Its last input line prints NVX-GUEST-SCRIPT-START and sources the script into the same shell, which keeps the semantics of typed lines, such as an exit that powers the guest off. No input is left to echo once the script runs. The result's new output field holds what the console printed after the start line, which the scenarios now parse, and the completion marker must be one of its lines, so that an echoed fragment of the staged script can stand for no marker. text keeps the whole transcript, whose status query lines precede the script and which the counting-LAPIC check still reads. _stage_script shares the here-document, which now refuses a script that holds its delimiter line or lacks a final newline. Every script now runs only once the guest has read it, so the comment that justified the exhaustive time ABI check's single input line goes. lifecycle.sh's bytes to the debug port now end with a newline: the next prompt of a typed script used to end their line, and the script's marker now follows them at once. The new tests drive the real harness against a fake console whose input arrives in pieces while commands run and whose terminal echoes each piece into the line being printed: typed line by line, the script reproduces CI's 'none0000 &', and staged, its output keeps the exact marker. The fakes of run_guest_script return the new field. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
After snapshot-core's no-destination source released its snapshot request, the console typed two lines at once: the time ABI status query, then the line that reports whether RCU stall detection is still suppressed. nvx-time status may wait up to 30 s for pending checks, and a slow guest could receive the second line while the query printed the status lines that check_canceled_capture parses, which the terminal would then echo that line into, as in #785. The checks now form one console line, which the shell reads whole before it runs any of it, so no input is left to echo while they print. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The guests of directional-network-policy, host-loopback-policy, private-network-ingress, and platform-network-policy announce that they listen and then read the host's release from the console. The release arrives while the script runs, so the terminal echoed it, and that echo could land inside a marker that the script printed afterwards, before the line's newline (#785). The host requires the completion markers of host-loopback-policy and private-network-ingress, and the probe report of platform-network-policy, as exact lines, which such an echo would hide. Each of these scripts now turns the terminal's echo off before it announces that it listens, and fails with a code of its own if it cannot. The read still receives the release, which no longer reaches the console. The tests that run two of these scripts in a local shell stub stty, since their input is a pipe there. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The CI guide explains why the test runners wait for the time ABI status query before any other input. Describe as well how they keep the guest terminal's echo out of the output of the scripts that follow (#785): the runners stage each script whole with a here-document before it runs, a scenario that parses its script's output reads only what follows NVX-GUEST-SCRIPT-START, and a script that waits for a line from the host turns the echo off first. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues were identified in the changes to prevent guest-console echo from corrupting microVM test markers.
No open issues 🎉
What changed in this PR
This PR addresses #785 by preventing guest-console input echo from corrupting markers parsed by NVX’s microVM test harness.
Changes:
- Stage scripts before execution and parse output after an explicit start marker, while preserving the full transcript for time ABI checks.
- Send canceled-capture checks as one console line, disable echo before host-release handshakes, and terminate lifecycle debug output with a newline.
- Add regression coverage for fragmented console input and document the staging behavior.
| File | Description |
|---|---|
scripts/test_microvm_tests.py |
Adds echo-interleaving regressions and updates script-result fixtures. |
scripts/nvx_tools/microvm_tests.py |
Stages scripts, isolates their output, and combines canceled-capture checks. |
scripts/nvx_tools/microvm_test_scripts/private-network-ingress.sh.in |
Disables echo before awaiting the host’s release. |
scripts/nvx_tools/microvm_test_scripts/platform-network-policy.sh.in |
Disables echo before the host-release handshake. |
scripts/nvx_tools/microvm_test_scripts/lifecycle.sh |
Ends debug-port output before the completion marker. |
scripts/nvx_tools/microvm_test_scripts/host-loopback-policy.sh.in |
Disables echo before announcing ingress readiness. |
scripts/nvx_tools/microvm_test_scripts/directional-network-policy.sh.in |
Disables echo before awaiting ingress confirmation. |
doc/ci.md |
Documents script staging, output boundaries, and echo suppression. |
Review effort: Balanced
Pedro Henrique Penna (ppenna)
deleted the
agents/issue-785-debug-test-solution
branch
October 29, 2026 02:35
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #785
test-microvmtyped each guest script into the guest's interactive shell line by line. The guest's terminal echoes console input that arrives while a command runs into whatever that command prints, so a guest that received a script in pieces could find the echo of its later lines inside the markers that its earlier lines printed. In run 39902391325,ipv6-egress-policy/allow-ipv6-tcp-rangereadnone0000 &as its DNS server: the uploaded transcript shows the tail of the UDP probe's line landing before the newline ofNVX-IPV6-EGRESS-DNS none. The scenarios that boot throughnvx-rs runalso skip the time ABI status handshake, because their command has no--kernel, so their script can reach the guest before its shell has even started; in that transcript the echo begins inside the shell'scan't access ttymessage.Changes
scripts: keep the console's echo out of guest script output:run_guest_scriptstores the script in the guest with a here-document, which the shell reads whole before it runs anything. Its last input line printsNVX-GUEST-SCRIPT-STARTand sources the script into the same shell, which keeps the semantics of typed lines, such as anexitthat powers the guest off. No input is left to echo once the script runs. The result gainsoutput, the console output after the start line, which the scenarios now parse, and the completion marker must be one of its lines.textkeeps the whole transcript, which the counting-LAPIC check needs for the status query lines that precede the script._stage_scriptshares the here-document, which now refuses a script that holds its delimiter line or lacks a final newline.lifecycle.shends its debug-port bytes with a newline, because its marker now follows them at once instead of after a prompt.scripts: type the canceled-capture checks as one console line:snapshot-coretyped thenvx-time statusquery and another line at once, and the second line could be echoed into the status lines thatcheck_canceled_captureparses.scripts: turn the echo off before guest scripts await the host:directional-network-policy,host-loopback-policy,private-network-ingress, andplatform-network-policyread the host's release from the console while they run. They now runstty -echobefore they announce that they listen, so the release cannot echo into the markers that follow, which the host matches as exact lines.doc: describe how the microVM tests keep the console's echo out:doc/ci.md.The regression tests drive the real harness against a fake console whose input arrives in pieces while commands run. Typed line by line, the CI-shaped script reproduces
NVX-IPV6-EGRESS-DNS none0000 &; staged, its output keeps the exact marker.Validation
On the
nvx-kvm-testLinux/KVM test VM (Azure Linux 3,linux-kvm,virtual-machine), with thev0.1.0-dev.5ea7b0cdffe1KVM package (OpenVMMb92daa350) andnvx-rsbuilt from the branch:Reproduction. Console input delivered in 24-byte pieces while commands run reproduces the CI transcript's condition. A 40-marker probe script typed line by line corrupted markers in 10 of 10 boots (30 of 400 marker lines, such as
NVX-ECHO-PROBE-026 nonealue:-none}"). Staged, it corrupted none in 10 boots.Fault injection with the real scenarios, two rounds each, comparing the harness of
1f9e0c794e43(the revision of the CI failure, with its own release and OpenVMM7ff31ed7a) against this branch:ipv6-egress-policydeny-ipv6-tcp-portread'fd00::c000:201hed'as its DNS servercommand-line-limitl3-l4-egress-policy,l3-l4-egress-protocols,l3-l4-egress-port-ranges,host-loopback-policy,private-network-ingress,directional-network-policy,snapshot-core,lifecycleFull default KVM suite with the change: every scenario passes except
filesystem-snapshotandplatform-aggregate-export, which need the OpenVMM pin from openvmm: let a platform restore attach an aggregate to a dormant filesystem slot #948 that the package predates. Running that suite caught thelifecycle.shregression, which is fixed in the first commit.ruff check,ruff format --check,pyright --pythonplatform LinuxandWindows, andscripts/check_markdown_links.py.test_microvm_tests.py,test_time_abi.py, andtest_nvx_tools.pyon Linux at the pushed head (877 tests). On Windows, the affected microVM and time ABI tests pass. On that Windows machine, sevenhost-loopback-policyandprivate-network-ingressscript subtests time out under Git'sshwith or without this change.Left to CI: the MSHV and WHP backends, and the full matrix with the current OpenVMM pin.