Skip to content

Tile libraries: call Rust device functions from Tile kernels - #1542

Closed
nihalpasham wants to merge 3 commits into
NVIDIA:mainfrom
nihalpasham:feat/tile-simt-interop
Closed

nihalpasham wants to merge 3 commits into
NVIDIA:mainfrom
nihalpasham:feat/tile-simt-interop

Conversation

@nihalpasham

Copy link
Copy Markdown
Collaborator

What this adds

A Rust device function can now be called from inside a Tile kernel. #[tile_export] marks the function, and cargo oxide tilelib packages the crate as a .tilelib, the static tile library format tileiras links with -l.

before: a Tile kernel and a Rust kernel could only share a stream and buffers
after:  a Tile kernel calls the Rust function in its body, one invocation per tile element
#[tile_export(shape = 128)]
pub fn relu(x: f32) -> f32 {
    if x > 0.0 { x } else { 0.0 }
}
cargo oxide tilelib tilelib_relu --arch sm_120   # -> tilelib_relu.tilelib

Direct linkage flow

cuda-oxide is the SIMT provider: it ships the code (LTOIR from the existing pipeline) and two small descriptions (the interface and the implementation, as Tile IR bytecode). tileiras does the matching, marshalling and linking. The Tile side only emits the call.

The contract

  • in scalars are passed by value, &T, &mut T and fixed arrays as pointers, and a scalar return value becomes a trailing out pointer. #[tile_export] generates that wrapper next to the function.
  • sync = "warp" means 32 lanes call the function together, so warp shuffles are legal inside it.
  • Preview content is read only at the assembler's current bytecode version. The command asks the installed tileiras and writes the descriptions at that version: 13.5 for a CUDA 13.5 assembler, 13.6 for the 13.6 one, and 13.5 when none can read them yet.
Implementation
  • cutile-ir writes the Tile IR 13.5 and 13.6 preview content: preview$func extern, preview$extern_impl, preview$call, fragment types and the symbolic layout attributes, behind a typed tilelib API, with a decoder and a structural verifier. The default version for ordinary kernels stays 13.4.
  • oxide-tile-export is the shared description vocabulary and its rules (one dimension, one footprint per export, written parameters as one contiguous run, whole warps for warp sync).
  • #[tile_export] generates the extern "C" wrapper plus a host-only metadata function whose body is the JSON description; the backend reads it from MIR, roots the wrapper, and writes <crate>.tileexports.json beside the device artifact.
  • cargo oxide tilelib runs emit-ltoir, checks that the NVVM IR defines every symbol with the parameters the ABI predicts, builds both bytecode halves, and calls tileiras --preview-create-tile-lib. On an older assembler it stops after the descriptions with exit 2, or exit 0 with --library-optional, which is how the smoketest's new tilelib lane runs it.
  • The tilelib_relu example has three consumers: a host program that proves the ABI on CUDA 13.4 (a CUDA C++ caller generated from the descriptions, linked with nvJitLink), a Tile C++ kernel (cpp/tile_caller.cu), and a cuTile Python kernel (python/call_from_cutile.py).

Verification

Checked at d1b988b2:

  • ✅ cutile-ir: 14 test suites (hand-built byte tests, decoder round trips, fixture text equal to the upstream Tile IR fixtures, and the preview$call bytes equal to cuTile Python's), clippy, rustdoc.
  • ✅ cargo-oxide 255 tests, cuda-macros 132 tests plus the trybuild pass and nine compile-fail cases, oxide-tile-export 16 tests; all 18 repository guards, lockfile sync, cargo deny, book with warnings as errors.
  • ✅ RTX 5090: cargo oxide run tilelib_relu verifies all three exports through the native ABI on CUDA 13.4.
  • ✅ RTX 5090 with the CUDA 13.6 development toolkit: cargo oxide tilelib builds tilelib_relu.tilelib, and the Tile C++ kernel calls all three exports through it (3 of 3, bit for bit).
  • ✅ cuTile Python 1.7 dry run: the kernel compiles to 13.5 bytecode carrying the preview$call to relu.

Caveats

  • The .tilelib itself needs a CUDA 13.5 or newer tileiras; on 13.4 the command writes the descriptions and the LTOIR and says so.
  • The cuTile Python run needs a cuTile Python that lists the assembler's bytecode version. The 1.7 release stops at 13.5, so it cannot drive the 13.6 development assembler yet; the Tile C++ path has no such gap.
  • One-dimensional tiles, f16 but not bf16, concrete shapes only. Linkage providers (implementations compiled on demand) are not part of this change.
  • cargo-oxide now depends on cutile-rs/cutile-ir by path, so the SIMT CI lanes also watch that directory.

Tile IR 13.5 adds the public preview `simt` feature: a Tile program can
call a native SIMT function from a tile library. This teaches cutile-ir
the content such a library is made of and how a caller reaches it:

- the ops `preview$func extern`, `preview$extern_impl` and
  `preview$call`, the fragment type and the symbolic layout attributes
  (fragmap, mem_layout, synchronization, access, scratch), with textual
  printing, decoding and structural verification;
- the 13.5 header tag and function table form, selected automatically
  when a module carries preview content; everything else still writes
  13.4, and negotiation with tileiras still stops there;
- a typed `tilelib` API (`ExternFunc`, `ExternImpl`, `interface_module`,
  `implementation_module`, `verify_library`, `preview_call`) that builds
  the two bytecode halves of a library without touching op payloads;
- `toolchain::list_bytecode_versions` and public names for the
  environment variables the toolchain lookup reads.

Checked against the reference implementation: the fixture modules print
like the upstream `.mlir` test inputs, the decoder round trips every new
op, and the `preview$call` encoding is byte-identical to what cuTile
Python's `foreign_call` emits for the same program. The compatibility
table in the book is regenerated for the new column.

Signed-off-by: nihalpasham <nihalp@nvidia.com>
…ile kernels

`#[tile_export(shape = N, ...)]` marks a device function as an entry of a
cuTile tile library. The attribute leaves the function as it is and
generates two items beside it: a `#[device] extern "C"` wrapper on the
native SIMT ABI the Tile compiler calls (`in` scalars by value, `&T`,
`&mut T`, `&[T; N]` and `&mut [T; N]` as pointers, a scalar return as a
trailing `out` pointer), and a host-only metadata function whose body is
the export's JSON description.

- `oxide-tile-export` (new crate): the description vocabulary shared by
  the macro, the backend and `cargo oxide tilelib`, with the rules a
  library entry must satisfy (one dimension, one footprint for every
  parameter, one contiguous run of written parameters, whole warps for
  `sync = "warp"`).
- `reserved-oxide-symbols`: the `cuda_oxide_tile_export_246e25db_`
  prefix of the metadata functions.
- `rustc-codegen-cuda`: the collector reads each description out of the
  metadata function's MIR, roots the wrapper even when the crate has
  kernels, rejects duplicate names and symbols and device-side calls of
  the metadata function, and writes `<crate>.tileexports.json` next to
  the device artifact.
- trybuild tests for the accepted forms and every rejection; the example
  lockfiles pick up the new dependency of cuda-macros.

Signed-off-by: nihalpasham <nihalp@nvidia.com>
…mple

`cargo oxide tilelib <crate> --arch sm_XX` packages a crate's
`#[tile_export]` functions as a cuTile tile library. It runs emit-ltoir,
checks that the NVVM IR defines every implementation symbol with the
parameter list the native SIMT ABI predicts, maps the descriptions to
cutile-ir's typed API, writes the Tile IR 13.5 interface and
implementation bytecode plus a readable `.tilelib.mlir`, and asks
`tileiras --preview-create-tile-lib` for the `.tilelib` when the
installed assembler lists bytecode 13.5. An older assembler stops after
the descriptions with exit 2, or exit 0 with `--library-optional`; the
summary ends with a "written to" or "skipped" line that scripts read.

The `tilelib_relu` example is a three-export library (`relu`,
`scale_in_place`, `warp_sum`). Its host program proves the ABI on the
installed toolkit: it generates a CUDA C++ caller from the descriptions,
compiles it to LTOIR with nvcc, links both halves with nvJitLink and
checks the numbers on the GPU. `python/call_from_cutile.py` is the cuTile
Python caller (`foreign_call`), with a dry run that stops at the bytecode.

Also: the smoketest `tilelib` lane, the book chapter on tile libraries,
CI path filters for `cutile-rs/cutile-ir`, and cargo-oxide's path
dependency on that crate.

Signed-off-by: nihalpasham <nihalp@nvidia.com>
@nihalpasham nihalpasham added the enhancement New feature or request label Oct 18, 2026
@nihalpasham
nihalpasham requested a review from roivanov as a code owner October 18, 2026 13:18
@nihalpasham nihalpasham added cargo-oxide The `cargo oxide` subcommand and build automation codegen Device code-generation pipeline (Rust MIR to IR to PTX) interop Interop with external compilers / frontends cutile-rs Tile programming model: cutile crates, compiler, macros, IR, kernels, and book labels Oct 18, 2026
@nihalpasham
nihalpasham deleted the feat/tile-simt-interop branch October 18, 2026 13:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cargo-oxide The `cargo oxide` subcommand and build automation codegen Device code-generation pipeline (Rust MIR to IR to PTX) cutile-rs Tile programming model: cutile crates, compiler, macros, IR, kernels, and book enhancement New feature or request interop Interop with external compilers / frontends

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant