Introducing CUDA Rust: Two Tracks for Writing GPU Kernels
NVIDIA announced two Rust-based GPU kernel programming tracks: cuda-oxide (SIMT-style) and cutile-rs (Tile-based), closing the gap that previously forced kernel authors to use CUDA C++ or Python cuda-oxide provides a custom rustc codegen backend compiling Rust directly to PTX via the Pliron IR framework and LLVM, requiring a pinned nightly toolchain and custom LLVM cutile-rs enables Tile-based GPU programming on stable Rust 1.89+ with CUDA 13.3 and no custom LLVM, and is already deployed in prod
Analysis
TL;DR
- NVIDIA announced two Rust-based GPU kernel programming tracks: cuda-oxide (SIMT-style) and cutile-rs (Tile-based), closing the gap that previously forced kernel authors to use CUDA C++ or Python
- cuda-oxide provides a custom rustc codegen backend compiling Rust directly to PTX via the Pliron IR framework and LLVM, requiring a pinned nightly toolchain and custom LLVM
- cutile-rs enables Tile-based GPU programming on stable Rust 1.89+ with CUDA 13.3 and no custom LLVM, and is already deployed in production by HuggingFace's Grout inference engine and mistral.rs
- Both projects enforce compile-time memory safety: cuda-oxide uses DisjointSlice and launch contracts to prevent aliasing, while cutile-rs uses tensor partitioning and ownership guarantees
- NVIDIA committed to growing CUDA Rust through 2027 and beyond, with planned inter-language interoperability between CUDA Rust, CUDA C++, and CUDA Python
Why It Matters
Rust is rapidly becoming the language of choice for AI systems infrastructure—drivers, inference engines, and serving layers—but GPU kernels have remained a C++/Python stronghold. CUDA Rust eliminates this last barrier, allowing developers to write performant, memory-safe kernels natively in Rust without relying on wrappers or foreign language bindings. This is a strategic move by NVIDIA to align GPU programming with the broader industry shift toward Rust in systems-level AI work.
Technical Details
- cuda-oxide is a custom
rustccodegen backend that intercepts compilation, routes#[kernel]functions through Rust MIR, the Pliron IR framework, and LLVM IR down to PTX, while passing non-kernel code to the standard backend; it requires Linux, a GPU with compute capability 8.0+, CUDA toolkit 12.x+, clang with libclang headers, and a pinned nightly toolchain (cargo +nightly-2026-04-03) - cutile-rs uses CUDA Tile IR JIT compilation to manage thread mapping and memory layout automatically, running on stable Rust 1.89+ with CUDA 13.3 and no custom LLVM; it is published on crates.io and already integrated into HuggingFace's Grout and mistral.rs
- Both tracks mirror CUDA's own dual programming model: SIMT (per-thread programming, drop-down for low-level control) and Tile (compiler-managed tile-level abstraction, recommended as the default starting point for new code)
- Memory safety is enforced at compile time in both projects: cuda-oxide uses
DisjointSliceandlaunch_contractattributes to prevent pointer aliasing, while cutile-rs leverages tensor partitioning and Rust ownership semantics to guarantee exclusive access - NVIDIA plans inter-language interoperability so code written in one CUDA Rust frontend can interoperate with CUDA C++ and CUDA Python ecosystems
Industry Insight
- The emergence of mature, memory-safe GPU kernel programming in Rust will likely accelerate adoption of Rust across the entire AI stack—from inference engines to serving infrastructure—reducing whole classes of memory bugs without sacrificing performance
- Developers should adopt the Tile-based approach (cutile-rs) as their default for new GPU code, reserving SIMT (cuda-oxide) for cases requiring fine-grained control over thread and memory layout, as the compiler-managed abstractions provide better portability across architectures
- NVIDIA's commitment to cross-language interoperability signals that CUDA Rust will not fragment the ecosystem; teams can invest in Rust kernel development without risking lockout from existing CUDA C++ or Python codebases, making this a low-risk strategic bet for organizations building on NVIDIA hardware
Disclaimer: The above content is generated by AI and is for reference only.