Nova: An End-to-End MLIR Compiler for Deep Learning Nova：一种端到端深度学习 MLIR 编译器

TL;DR

• Nova is an automated end-to-end JIT compiler built on MLIR that captures eager executions and unifies forward/backward passes into a single value-semantic dialect for aggressive whole-graph optimization
• An Analytic Configurator derives optimal execution schedules deterministically based on arithmetic intensity, eliminating search time entirely
• A structural hashing runtime enables fine-grained kernel synthesis directly from computation structure, achieving up to 29% memory reduction versus PyTorch
• On an RTX 3060, Nova matches or exceeds cuBLAS and XLA on TF32 matmuls with < 5e-4 relative error, and delivers up to 10.6% greater throughput than PyTorch and 4.4% greater than XLA on a 42M-parameter model
• Nova successfully trains a 144M-parameter model at 17,900 tokens/s on a 12 GB consumer GPU where PyTorch fails with OOM errors

Why It Matters

Nova addresses a critical bottleneck in deep learning: the gap between high-level framework abstractions and low-level hardware utilization. By providing whole-graph visibility and deterministic optimization without search overhead, it offers a practical path for practitioners to extract maximum performance from consumer-grade hardware, potentially democratizing access to efficient model training.

Technical Details

• Nova captures eager PyTorch-style executions and compiles them into a unified value-semantic MLIR dialect that encompasses both forward and backward passes, enabling cross-boundary operation fusion and memory hierarchy optimization
• The Analytic Configurator uses arithmetic intensity as a sole decision metric to deterministically derive optimal execution schedules, reducing compilation/search time to zero compared to traditional auto-tuning approaches
• A structural hashing runtime maps computation structures to synthesized fine-grained kernels, providing granular control over hardware mapping from operation fusion down to register-level tuning
• Evaluated on an RTX 3060 (12 GB VRAM), Nova was benchmarked against cuBLAS, XLA, and PyTorch on TF32 matmuls across various shapes, maintaining numerical fidelity with relative error below 5e-4
• At the model level, Nova achieved up to 10.6% throughput improvement over PyTorch and 4.4% over XLA on a 42M-parameter model, while reducing memory footprint by up to 29% relative to PyTorch

Industry Insight

• The deterministic Analytic Configurator approach challenges the prevailing auto-tuning paradigm, suggesting that arithmetic-intensity-based scheduling could eliminate compilation latency—a significant advantage for iterative development workflows and dynamic model architectures
• Nova's ability to train larger models on consumer hardware (144M params on 12 GB vs. PyTorch OOM) demonstrates that compiler-level memory optimization can materially extend the viability of affordable GPUs for research and production, reducing dependency on datacenter-class hardware
• The integration of forward/backward unification within a single MLIR dialect represents a compelling architectural pattern for future compilers, particularly as memory efficiency becomes increasingly critical with growing model sizes and the rise of on-device AI inference

TL;DR

• Nova是一个端到端JIT编译器，通过捕获eager执行并将前向/后向传递统一为单一值语义dialect，实现激进的整图优化
• 采用分析配置器（Analytic Configurator）基于算术强度确定性地推导最优执行调度，将搜索时间降至零
• 在RTX 3060上，Nova在TF32 matmuls上匹配或略超cuBLAS和XLA，保持<5e-4的严格相对误差
• 相比PyTorch，Nova在42M参数模型上实现10.6%吞吐量提升，内存占用减少29%
• 成功在12GB消费级GPU上训练144M参数模型（17,900 tokens/s），而PyTorch在此配置下OOM失败

为什么值得看

Nova为深度学习框架的eager执行模式提供了整图优化能力，解决了传统编译器缺乏硬件级控制的问题。其确定性调度策略消除了搜索开销，为消费级GPU上的大规模模型训练提供了可行方案。

技术解析

Nova的核心创新在于将eager执行捕获并转换为统一的前向/后向值语义dialect，从而获得整图可见性以执行跨算子融合和内存层次优化。分析配置器基于算术强度直接计算最优执行调度，避免了传统编译器中耗时的搜索过程。

结构哈希运行时支持从计算结构直接合成细粒度内核，实现对硬件映射的绝对控制，包括寄存器级调优。

在RTX 3060上的基准测试显示，Nova在TF32矩阵乘法上匹敌或超越cuBLAS和XLA，同时保持严格的数值精度（<5e-4相对误差）。

模型级评估表明，Nova在42M参数模型上比PyTorch吞吐量高10.6%、比XLA高4.4%，内存占用降低最多29%。

行业启示

深度学习编译器正从静态图向支持eager执行的动态图演进，Nova证明了在保持开发灵活性的同时实现硬件级优化的可行性。

消费级GPU训练大规模模型成为可能，Nova通过内存优化使12GB显存成功训练144M参数模型，降低了AI研究的硬件门槛。

确定性调度策略消除了编译搜索开销，为实时JIT编译场景提供了新的设计范式，值得工业界借鉴。