Repository navigation
Failure to run PGO instrumentation on Rust 1.99.0 #163616
Copy link
Copy link
Closed
Labels
A-LLVMArea: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues.Area: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues.A-PGOArea: Profile-guided optimizations (PGO)Area: Profile-guided optimizations (PGO)C-bugCategory: This is a bug.Category: This is a bug.S-has-bisectionStatus: A bisection has been found for this issueStatus: A bisection has been found for this issueT-compilerRelevant to the compiler team, which will review and decide on the PR/issue.Relevant to the compiler team, which will review and decide on the PR/issue.llvm-fixed-upstreamIssue expected to be fixed by the next major LLVM upgrade, or backported fixesIssue expected to be fixed by the next major LLVM upgrade, or backported fixesregression-from-stable-to-stablePerformance or correctness regression from one stable version to another.Performance or correctness regression from one stable version to another.
Description
Code
I tried this code (disclosure: this was reduced from a real-world project using an AI agent, I have verified this reproduction indeed fails 1.99.0 and works fine on 1.98.1):
// repro.rs
use std::hint::black_box;
/// Reproduction needs two enum variants - `Words` has data which
/// overlaps a nested discriminant in the other variant.
enum Value {
#[expect(dead_code)]
Optional(Option<u64>),
Words([u64; 2]),
}
#[inline(never)]
fn select(value: &Value) -> u64 {
match value {
Value::Optional(Some(_)) => 42,
Value::Optional(None) => 7,
Value::Words(words) => words[0],
}
}
/// AI claims a single call to `select` incorrectly adds the `Words` payload
/// to a PGO counter on Rust 1.99.
fn main() {
let value = black_box(Value::Words([1 << 60, 0]));
assert_eq!(select(&value), 1 << 60);
}I wanted to compile it and run PGO instrumentation:
rustc +1.99 -O -Cprofile-generate repro.rs -o repro
LLVM_PROFILE_FILE=repro.profraw ./repro
"$(rustc +1.99 --print target-libdir)/../bin/llvm-profdata" \
merge -o repro.profdata repro.profrawI expected to see this happen: PGO instrumentation runs and merges the profile successfully
Instead, this happened:
warning: repro.profraw: excessively large counter value suggests corrupted profile data: 1152921504606846976
error: no profile can be merged
Version it worked on
It most recently worked on: Rust 1.98.1
(replace +1.99 with +1.98 above in the compilation steps and it runs fine)
Version with regression
rustc --version --verbose:
rustc 1.99.0 (b940084d7 2026-09-28)
binary: rustc
commit-hash: b940084d7eb6a299eb4bfeb8e34901bc051e7ac4
commit-date: 2026-09-28
host: x86_64-unknown-linux-gnu
release: 1.99.0
LLVM version: 23.1.1
@rustbot modify labels: +regression-from-stable-to-stable -regression-untriaged
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
A-LLVMArea: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues.Area: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues.A-PGOArea: Profile-guided optimizations (PGO)Area: Profile-guided optimizations (PGO)C-bugCategory: This is a bug.Category: This is a bug.S-has-bisectionStatus: A bisection has been found for this issueStatus: A bisection has been found for this issueT-compilerRelevant to the compiler team, which will review and decide on the PR/issue.Relevant to the compiler team, which will review and decide on the PR/issue.llvm-fixed-upstreamIssue expected to be fixed by the next major LLVM upgrade, or backported fixesIssue expected to be fixed by the next major LLVM upgrade, or backported fixesregression-from-stable-to-stablePerformance or correctness regression from one stable version to another.Performance or correctness regression from one stable version to another.
Activity
lqd commentedon Oct 1, 2026
Does this reproduce on nightly or beta? A bisection would be helpful (https://github.com/rust-lang/cargo-bisect-rustc).
Fayti1703 commentedon Oct 1, 2026
I ran a bisection on this -- looks like this broke during an LLVM update (#158734).
Raw result:
searched nightlies: from nightly-2026-05-22 to nightly-2026-10-01
regressed nightly: nightly-2026-08-06
searched commit range: 1ed2df6...7608eb7
regressed commit: 1b29cfd
bisected with cargo-bisect-rustc v0.6.11
Host triple: x86_64-unknown-linux-gnu
Reproduce with:
Where
check-repro.shis:and
repro.rsis (from the issue comment):theemathas commentedon Oct 1, 2026
cc @nikic
theemathas commentedon Oct 1, 2026
By the way, thanks OP for summarizing what I assume is verbose AI output.
davidhewitt commentedon Oct 1, 2026
All good; I get too many issues and PRs to PyO3 filled with AI prose and it's always extremely annoying (and a waste of my time reading the fluff), so I have no intention to ever push noise upon others. I am also aware that LLM use is extremely contentious within
rust-lang, PyO3's policy is slightly more permissive but I certainly do not wish PyO3 to become slop.For what it's worth, the agent also found a workaround in
-Cllvm-args=-pgo-instr-select=false- which I assume is disabling counters for this particular case (at loss of PGO quality?).-Copt-level=0also doesn't hit the problem, so I guess it's something to do with an optimization in LLVM?I'll see if I can turn this into something to report against LLVM.
davidhewitt commentedon Oct 4, 2026
I have also reported this against LLVM as llvm/llvm-project#228895
Auto merge of #164254 - nikic:llvm-update-4, r=cuviper
Auto merge of #164254 - nikic:llvm-update-4, r=cuviper
Auto merge of #164254 - nikic:llvm-update-4, r=cuviper