Skip to content

fix(mir-lower): cap enum alignment anchors - #1357

Merged
nihalpasham merged 6 commits into
NVIDIA:mainfrom
YunusMutlu:fix/enum-alignment-anchor
Oct 19, 2026
Merged

nihalpasham merged 6 commits into
NVIDIA:mainfrom
YunusMutlu:fix/enum-alignment-anchor

Conversation

@YunusMutlu

@YunusMutlu YunusMutlu commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

What this adds

When mir-lower mints the LLVM storage type of an enum whose tag and payload bytes under-align it, it inserts a zero-length integer array at slot 0 whose element width encodes the alignment. The width was align * 8 with no bound, so a #[repr(align(1 << 29))] enum wrapped to i0, which LLVM rejects, and anything from repr(align(1 << 21)) up already exceeded LLVM's integer width limit. The anchor is now capped at 16 bytes, [0 x i128], the cap union storage has used since #610. The Rust alignment still rides on every alloca, load and store of the enum.

#[repr(align(536870912))] pub enum HugeAlign { A, B }

before: { [0 x i0], i8, [536870911 x i8] }     // llc: bitwidth out of range, exit 101
after:  { [0 x i128], i8, [536870911 x i8] }   // builds, runs; load i8 ... align 536870912
// one helper for unions and enums, in mir-lower's layout module
pub(crate) const MAX_NATURAL_ALIGN: u64 = 16;
pub(super) fn alignment_anchor_type(ctx: &mut Context, abi_align: u64) -> TypeHandle

The enum alignment anchor

What we fixed

  • Over-aligned enums (repr(align(2^20)), 2^21, 2^29) go from a failed build to building, running and passing memcheck. Enums aligned 32, 64 and 128 produce instruction-identical PTX.
  • One helper builds the anchor for unions and enums, with the 16-byte cap as a named constant and the reason stated: the exporter's data layout strings, the layout model and the exporter's natural alignment all stop at i128, while llc's own NVPTX layout gives i256 a 32-byte alignment, so a wider anchor made opt and llc disagree and bought nothing.
  • A lowering-level regression beside the union one, a 2^21 table row, and a repr(align(32)) enum case in the repr_u32_enum_stride example so an over-aligned enum runs on a GPU in CI.
Implementation
  • convert/types/layout.rs: MAX_NATURAL_ALIGN and alignment_anchor_type, used by union_storage.rs, enum_layout.rs, the layout model and the test fixtures. The size, the stride and the alignment stamped on memory operations do not change; only the anchor's element width for alignments of 32 and up.
  • Tests: the alignment table (8 and 16 as controls, 32 to 128, 2^20, 2^21, 2^29), and convert_ref_preserves_over_aligned_enum_array_layout_and_alignment over 32, 64 and 2^29.

Verification

Checked at e8e81bc4e (nightly-2026-10-11, RTX 5090):

  • ✅ cargo test -p mir-lower --all-targets: 322 unit and 147 integration tests pass; 2440 downstream tests in mir-importer and cuda-oxide-codegen pass. Clippy and rustdoc with warnings as errors, fmt, the example's fmt and clippy gates, the 18 guards and the lock check: clean.
  • ✅ Fails before, passes after: with the anchor unbounded, the table test stops at the 32-byte row (anchor 256, expected 128) and the lowering test at 2^29 (natural alignment 1, expected 16).
  • ✅ cargo oxide run repr_u32_enum_stride: align32_enum [0, 1, 1, 0] PASS, memcheck clean. Scratch kernels with enums aligned 2^20, 2^21 and 2^29 fail to build on main and run here.
  • ✅ Raw-pointer, local-array, slice, by-value, niche, nested-payload and shared-memory kernels over align 32, 64 and 128 enums: PTX identical to main.

Caveats

Closes #1356.

@nihalpasham nihalpasham added the cuda-oxide SIMT programming model: rustc backend, cargo oxide, cuda-device, cuda-host, book label Oct 11, 2026
Signed-off-by: Yunus Mutlu <mutluyunus708@gmail.com>
@YunusMutlu
YunusMutlu force-pushed the fix/enum-alignment-anchor branch from af0448f to ffa23db Compare October 13, 2026 12:00
@copy-pr-bot

copy-pr-bot Bot commented Oct 13, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

…enums

Union and enum storage each built their own zero-length alignment anchor
and each capped it at a bare 16. One helper, alignment_anchor_type, now
builds the anchor for both, and MAX_NATURAL_ALIGN names the cap for the
layout model and the test fixtures as well.

The helper's doc states the real reason for the cap: the exporter's data
layout strings, llvm_type_size_align and the exporter's natural_alignment
all stop at i128, while llc's default NVPTX layout has i256:256, so a
wider anchor made opt and llc disagree on a type's alignment and raised
nothing. Rust alignment is stamped on the memory operations. The three
comments that said NVPTX itself stops at 16 bytes now point at the helper.

Tests: the enum anchor table gains the 2^21 row (the first width LLVM
rejects; 2^29 is the u32 wrap), and a lowering-level regression beside
the union one checks that an over-aligned enum array keeps the Rust
alignment on the alloca and the store at 32, 64 and 2^29 while the
storage keeps rustc's size and stride. Both fail with an unbounded
anchor.

Signed-off-by: nihalpasham <nihalp@nvidia.com>
No tracked example read an over-aligned enum on a GPU, so the capped
alignment anchor had no end-to-end coverage. Tag32 is a u8 tag padded to
32 bytes; the new kernel reads it through *const Tag32 over a byte
buffer, so the 32-byte stride and the align 32 on the tag load both have
to come from rustc's layout. The host checks tags at offsets 0, 32, 64
and 96. The PASS marker, manifest, lockfile and smoketest category are
unchanged.

Signed-off-by: nihalpasham <nihalp@nvidia.com>
The debug assertion ran after the clamp, so it could only ever see 1 to 16
and never caught a non-power-of-two ABI alignment above the cap.

Signed-off-by: nihalpasham <nihalp@nvidia.com>
A one-page figure for the book and the pull request, in the style of the
tile-libraries direct-linkage figure.

Signed-off-by: nihalpasham <nihalp@nvidia.com>
@nihalpasham nihalpasham added bug Something isn't working IR-lowering Lowering between dialects (dialect-mir to LLVM dialect) codegen Device code-generation pipeline (Rust MIR to IR to PTX) labels Oct 19, 2026
@nihalpasham

Copy link
Copy Markdown
Collaborator

/ok to test e8e81bc

@nihalpasham

Copy link
Copy Markdown
Collaborator

Thanks, this is the right fix and it matches what union storage has done since #610. On top:

  • One helper now builds the anchor for unions and enums, with the 16-byte limit as a named constant and the real reason stated: our data layout strings and the exporter stop at i128, while llc's own NVPTX layout gives i256 a 32-byte alignment, so a wider anchor made opt and llc disagree and bought nothing.
  • Added the lowering-level regression beside the union one, the 2^21 row (from there up the old anchor exceeded LLVM's integer width limit, a second failure before the i0 wrap at 2^29), and a repr(align(32)) enum in repr_u32_enum_stride so an over-aligned enum runs on a GPU in CI.

At e8e81bc4e: mir-lower unit and integration tests, clippy, rustdoc, the guards and all CI checks passed. On the 5090, enums aligned 2^20, 2^21 and 2^29 go from a failed build on main to running with memcheck clean, and the PTX for align 32, 64 and 128 is instruction-identical to main. That takes care of #1356.

@nihalpasham
nihalpasham merged commit 00426c8 into NVIDIA:main Oct 19, 2026
64 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working codegen Device code-generation pipeline (Rust MIR to IR to PTX) cuda-oxide SIMT programming model: rustc backend, cargo oxide, cuda-device, cuda-host, book IR-lowering Lowering between dialects (dialect-mir to LLVM dialect)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Over-aligned enums generate invalid i0 LLVM IR

3 participants