Skip to content

fix(cargo-oxide): pin release-like codegen in Cargo profiles - #1510

Merged
nihalpasham merged 5 commits into
NVIDIA:mainfrom
ssoj13:nvidia/fix/release-profile-pins
Oct 19, 2026
Merged

nihalpasham merged 5 commits into
NVIDIA:mainfrom
ssoj13:nvidia/fix/release-profile-pins

Conversation

@ssoj13

@ssoj13 ssoj13 commented Oct 13, 2026 •

Copy link
Copy Markdown

What this adds

Release-like cargo oxide routes used to force opt-level=3, debug-assertions=off and (with --debug-assertions) overflow-checks=off through CARGO_ENCODED_RUSTFLAGS. Cargo hashes rustflags into every crate's fingerprint, while cargo oxide test -- --release added none, so the first switch between build and test in each direction rebuilt the whole dependency graph and left two copies of every artifact. The pins now travel in the Cargo profile (CARGO_PROFILE_RELEASE_*, CARGO_PROFILE_DEV_*, and --config profile.<name>.* for a selected custom profile), which Cargo compares by value, so both commands share every compiled crate.

cargo oxide test -- --release        # compiles the dependency graph (72 crates)
cargo oxide build -- --release

Before: Compiling cuda-core ... 72 crates recompiled, 13 s, second artifact set
After:  1 crate recompiled (the application), 71 reused, 0.4 s, PTX unchanged
cargo oxide build -- --release                 # pins via CARGO_PROFILE_RELEASE_*
cargo oxide build -- --profile release-lto     # pins via --config profile.release-lto.*

What we fixed

  • Build and test -- --release passthrough commands hand rustc the same CARGO_ENCODED_RUSTFLAGS; the pins move to Cargo profile settings at the single site every release-like route uses (example build/run/pipeline/debug, emit-ltoir, tilelib, interop device crates, sanitize host binary, passthrough build).
  • overflow-checks=off is pinned explicitly in every release-like policy. On main only the --debug-assertions policy set it, so an incoming -C overflow-checks=on or [profile.release] overflow-checks = true reached device code, and a kernel with a reachable overflow trapped at runtime.
  • A -C pin is still appended after incoming rustflags that set the same option (-C x, -Cx, --codegen, -O, -g, underscore forms), so those still lose to the pin.
  • A dashed custom --profile such as release-lto no longer produces CARGO_PROFILE_RELEASE_LTO_*, which Cargo parsed as profile.release.lto and rejected before compiling; custom profiles receive the pins as --config arguments.
  • cargo oxide debug gets debug = 2 from the same pinned list instead of a hand-set environment variable; the cargo oxide pipeline hint lists all three pinned settings.

Contract

  • Pins are profile values. [profile.*.package.*] overrides, --config profile.* and manifest [profile.*.rustflags] / [profile.*.package.*.rustflags] outrank them; the trailing -C re-pin covers RUSTFLAGS, CARGO_ENCODED_RUSTFLAGS and extra-rustflags only.
  • cargo oxide test leaves the selected Cargo profile alone, as before.
  • Build scripts and proc-macros follow Cargo's build-override settings instead of the old -Copt-level=3.
Implementation
  • CodegenProfilePolicy::pinned_options is the one list of pins; each PinnedOption pairs the rustc -C option with its Cargo manifest key and value, and derives the environment key by Cargo's uppercase-and-underscore rule.
  • apply_profile_pins sets CARGO_PROFILE_RELEASE_* and CARGO_PROFILE_DEV_*; cargo_passthrough_command adds --config profile.<name>.<key>=<value> right after the subcommand when --profile names a profile other than release or dev, before the user arguments, so a later user --config still wins.
  • passthrough.rs carries the --config pins for a selected custom profile; codegen_option_names recognises every rustc form of a codegen option; build_encoded_rustflags_with_existing appends a pin flag only for options the incoming flags set, after -Zcodegen-backend and before the -Z invariants.
  • Tests: identical Build/Test passthrough rustflags, the option-name parser, base and custom profile carriers, and real-Cargo freshness probes (build, test, build reuses the shared dependency; a dashed custom profile builds with the pins in force).

Verification

Checked at 90883b87a merged with main 5b606fddd (nightly-2026-10-11, RTX 5090, CUDA 13.4):

  • ✅ cargo test -p cargo-oxide --all-targets: 268 passed, the two real-Cargo probes among them (build, test, build reuses the shared dependency; a dashed custom profile builds with the pins in force). Clippy and rustdoc with -D warnings, cargo oxide fmt --check, the lock check and the 18 guards pass. All 54 CI checks pass on the PR; they ran on 90883b87a merged with main 82f195bea, and the merge with 5b606fddd is the local run above.
  • ✅ cargo oxide run vecadd passes and its PTX is byte-identical to main's, from the PR's pipeline and from main's own cargo-oxide; the passthrough build at sm_80 is byte-identical as well.
  • ✅ Reuse: cargo oxide test -- --release followed by cargo oxide build -- --release recompiles only the application crate (1 of 72); on main the same sequence recompiles all 72.
  • ✅ Custom profile: cargo oxide build -- --profile release-lto on a copy of vecadd with opt-level = 1 in that profile builds with -C opt-level=3 in force; the author's head carried the pins as CARGO_PROFILE_RELEASE_LTO_*, which Cargo rejects before compiling (error in environment variable CARGO_PROFILE_RELEASE: could not load config key profile.release, exit 101, reproduced with plain Cargo on nightly-2026-10-11). compute-sanitizer memcheck on the release and release-lto binaries: 0 errors.
  • ✅ cargo oxide pipeline vecadd prints the three pinned settings.
  • ✅ Full GPU smoketest on the tree that stacks this PR with fix(cuda-oxide): keep host-written constant memory through LTO #1509 and feat(cuda-oxide): carry inline intent through the device pipeline #1511: every lane that touches cargo-oxide passes (see feat(cuda-oxide): carry inline intent through the device pipeline #1511 for the one example fixed by test(drop_glue): keep destructor MIR with -Zinline-mir=no, not #[inline(never)] #1549).

Caveats

  • A kernel built with overflow checks or below opt-level = 3 through a package override, --config profile.* or manifest profile rustflags gets no diagnostic yet; a backend-side check is a follow-up.
  • Build scripts and proc-macros compile at build-override opt-level = 0 instead of 3; device code is unaffected.
  • A user -Copt-level=3 in RUSTFLAGS still gets a trailing re-pin on build but not on test, so those two stop sharing; cache efficiency only.
  • A profile name with a non-ASCII letter is not a TOML bare key and receives no --config carrier; it keeps only the base pins it inherits.
  • The author's 232/234 smoketest and just check figures come from the port PR on WSL2, not from this branch; here the examples-compile workflow and the stacked GPU smoketest cover it.

Port of ansidium#12.

Closes #1508.

…rustflags

Release-like routes (`build`, `run`, `debug`, interop, passthrough `build`)
appended `-Copt-level=3 -Cdebug-assertions=off` (plus overflow-checks /
debuginfo variants) to CARGO_ENCODED_RUSTFLAGS, while `test` leaves the
profile alone. The values equal the release defaults, but different rustflags
change every crate's fingerprint: `cargo oxide test -- --release` followed by
`cargo oxide build` recompiled the whole dependency graph (~680 crates for one
application) and CI caches held two copies of everything.

`CodegenProfilePolicy::pinned_options` is now the single list of pins. They
are set as `CARGO_PROFILE_{RELEASE,DEV,<--profile>}_*`; Cargo fingerprints
resolved profile values, so build and test share every artifact. A pinned
`-C` flag is still appended after incoming rustflags that set the same option
(including `-O`, `-g` and `_` spellings), so the pin keeps winning. Every
release-like policy now pins `overflow-checks = off` explicitly: a passthrough
build without `--release` runs the dev profile, whose overflow-checks default
is on. `cargo oxide debug`'s manual CARGO_PROFILE_RELEASE_DEBUG moved into the
same list. `[profile.*.package.*]` overrides and `--config profile.*` outrank
the environment; the README says so.

Signed-off-by: Alex Khal <joss13@gmail.com>
Signed-off-by: ssoj13 <96214515+ssoj13@users.noreply.github.com>
@ssoj13
ssoj13 requested a review from nihalpasham as a code owner October 13, 2026 19:29
@copy-pr-bot

copy-pr-bot Bot commented Oct 13, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@nihalpasham nihalpasham added the cuda-oxide SIMT programming model: rustc backend, cargo oxide, cuda-device, cuda-host, book label Oct 15, 2026
A `--profile` other than `release` or `dev` received its pins through
`CARGO_PROFILE_<NAME>_*`. For a dashed name such as `release-lto` Cargo
reads that key as `profile.release.lto` and stops before compiling, so a
build that worked through rustflags became a hard error. The environment
now carries the pins for the two base profiles only, and a selected
custom profile receives them as `--config profile.<name>.<key>=<value>`
arguments placed right after the subcommand. Cargo compares the resolved
values, so a build and a test under the same custom profile keep sharing
artifacts, and a later user `--config` for the same key still wins.

PinnedOption stores the manifest key and derives the environment key the
way Cargo does. Unit tests cover both `--profile` forms, the base
profiles, the test route, the debug-assertions policy and a name Cargo
would reject.

Signed-off-by: nihalpasham <nihalp@nvidia.com>
Extend the freshness probe to take the passthrough subcommand and extra
Cargo arguments, and key a test harness apart from the library it is
built from. Two probes drive real Cargo on a scratch workspace: a release
build followed by `test --release` reuses the shared dependency, and a
dashed custom profile that sets `opt-level = 1` and overflow checks
builds in both `--profile` forms with the pins in force.

Through rustflags the first probe recompiled the dependency. At the
previous head the second stopped on the rejected environment key, and
without the explicit overflow-checks pin the binary trapped.

Signed-off-by: nihalpasham <nihalp@nvidia.com>
Name the environment carrier for `release` and `dev` and the `--config`
carrier for a selected `--profile`. Scope the trailing `-C` re-pin to
RUSTFLAGS, CARGO_ENCODED_RUSTFLAGS and `extra-rustflags`, and list
manifest profile rustflags next to the package overrides and `--config`
arguments that outrank the pins. The `cargo oxide pipeline` hint now
lists `overflow-checks=off`, and the run hint mentions the `--config`
carrier.

Signed-off-by: nihalpasham <nihalp@nvidia.com>
@nihalpasham

Copy link
Copy Markdown
Collaborator

Thanks for this, the fix is right and it is going in. One snag: your fork is a fork of ansidium/cuda-rust-windows, not of NVIDIA/cuda-rust, so "allow edits from maintainers" gives push rights to ansidium, not to us, and I could not push my follow-ups onto your branch.

I reviewed and validated this on the 5090 and added the follow-ups on top of your commit, on branch landing/pr-1510 in https://github.com/nihalpasham/cuda-rust. Could you fork NVIDIA/cuda-rust directly, push that branch to your fork, and open the PR from there? Then it lands with you as the author. I will close this one once the new PR is up.

@ssoj13

ssoj13 commented Oct 18, 2026 •

Copy link
Copy Markdown
Author

Thanks @nihalpasham! Since GitHub only permits one fork per repository network (ssoj13/cuda-rust-windows is already in this fork network via ansidium/cuda-rust-windows), I fetched your exact landing/pr-1510 branch (90883b87a6fc3141066969346729e281281a369f) and pushed it directly onto this PR's head branch (nvidia/fix/release-profile-pins), as well as ssoj13/cuda-rust-windows:landing/pr-1510. Ready to merge!

@nihalpasham

Copy link
Copy Markdown
Collaborator

/ok to test 90883b8

@nihalpasham nihalpasham added bug Something isn't working cargo-oxide The `cargo oxide` subcommand and build automation build-related Build scripts, toolchain discovery, or compilation setup labels Oct 19, 2026
@nihalpasham

nihalpasham commented Oct 19, 2026 •

Copy link
Copy Markdown
Collaborator

Good catch, this one annoyed me too, and thanks for carrying the branch over. Landed with a few follow-ups:

  • --profile release-lto broke on this branch: the pins became CARGO_PROFILE_RELEASE_LTO_*, which Cargo reads as profile.release.lto and rejects. Custom profiles now get their pins as --config profile.<name>.<key>=<value>; release and dev keep the environment carrier.
  • Added two real-Cargo tests: build, test, build shares every dependency, and a dashed custom profile builds with the pins in force.
  • The docs now say what outranks the pins (package overrides, --config profile.*, manifest profile rustflags) and what the trailing -C re-pin still beats (RUSTFLAGS, CARGO_ENCODED_RUSTFLAGS, extra-rustflags).
  • cargo oxide pipeline lists overflow-checks=off with the other two pins.

On the 5090 the switch from cargo oxide test -- --release to cargo oxide build -- --release recompiles 1 crate instead of 72, and the PTX is identical to main for every policy I tried. Bonus: an incoming -C overflow-checks=on used to reach device code on main (a scratch kernel with a reachable overflow trapped); the explicit pin stops that. At 90883b87a merged with today's main the cargo-oxide suite (268 tests) and the guards are green locally, and all 54 CI checks pass on the PR. That takes care of #1508.

@nihalpasham
nihalpasham merged commit f6507e7 into NVIDIA:main Oct 19, 2026
60 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working build-related Build scripts, toolchain discovery, or compilation setup cargo-oxide The `cargo oxide` subcommand and build automation cuda-oxide SIMT programming model: rustc backend, cargo oxide, cuda-device, cuda-host, book

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: release-like rustflags invalidate build and test artifact reuse

2 participants