CAKE: Compiler-Agent Co-Design for Frontier Kernel Evolution
CAKE introduces a compiler-agent co-design where AI agents author CAKE IR, a typed, hardware-explicit intermediate representation that exposes warp roles, memory movement, synchronization, and pipelines The system treats the compiler as a co-design partner rather than a black box, enabling agents to make informed scheduling decisions instead of relying solely on error messages and timing feedback On B200 GPUs, CAKE-generated Flash-KMeans outperforms both direct CUDA/PTX (1.144x vs 0.928x relativ
Analysis
TL;DR
- CAKE introduces a compiler-agent co-design where AI agents author CAKE IR, a typed, hardware-explicit intermediate representation that exposes warp roles, memory movement, synchronization, and pipelines
- The system treats the compiler as a co-design partner rather than a black box, enabling agents to make informed scheduling decisions instead of relying solely on error messages and timing feedback
- On B200 GPUs, CAKE-generated Flash-KMeans outperforms both direct CUDA/PTX (1.144x vs 0.928x relative to tuned FlashML baseline) and achieves a 2.05x geometric-mean speedup over official FlashKDA for Kimi Delta Attention
- The harness is self-evolving: recurring failures automatically become verifier rules, IR primitives, model calibrations, and reusable optimization tactics
- CAKE supports NVIDIA GPUs from Ampere through Blackwell, with dispatcher-backed KNN and KMeans improving performance by 1.42x–2.12x across 400+ shapes and four kernels submitted as upstream PRs
Why It Matters
CAKE bridges a critical gap between AI agent capabilities and low-level GPU kernel optimization, demonstrating that agents can produce expert-level kernels when given the right hardware-explicit representation. This approach could democratize high-performance kernel development, reducing reliance on scarce GPU programming expertise while pushing performance beyond what traditional manual tuning achieves.
Technical Details
- CAKE IR: A typed, hardware-explicit schedule representation that exposes warp roles, memory movement patterns, synchronization points, and pipeline structures—giving agents visibility into decisions typically hidden by compilers or abstracted away by DSLs
- Compiler-agent co-design: Unlike prior approaches where agents treat compilers as black boxes receiving only errors and timing, CAKE integrates the compiler into the agent loop, supporting verification, cost modeling, and localized diagnostics
- Self-evolving harness: The system learns from recurring failures by automatically converting them into verifier rules, new IR primitives, model calibrations, and reusable optimization tactics, creating a compounding improvement loop
- Benchmark results: Flash-KMeans on B200 at 80M tokens achieves 1.144x tuned FlashML baseline (vs 0.928x for CUDA/PTX); Kimi Delta Attention gets 2.05x geometric-mean speedup over FlashKDA; KNN/KMeans dispatcher covers 400+ shapes with 1.42x–2.12x improvement
- Hardware coverage: Targets NVIDIA GPUs from Ampere through Blackwell, with a clean separation between single-shape evolution and library generalization/dispatch
Industry Insight
- The CAKE paradigm signals a shift toward AI-driven systems programming, where agents can autonomously produce production-grade kernels—potentially reducing the GPU expert bottleneck that constrains ML infrastructure scaling
- The self-evolving harness design offers a blueprint for other domains where iterative refinement against hardware feedback is needed, suggesting that "learning from failure" at the compiler level can compound into significant performance gains
- With four kernels already accepted as upstream PRs, CAKE demonstrates that agent-generated code can meet production quality standards, making it a viable path for organizations to accelerate kernel development without proportional increases in specialized staffing
Disclaimer: The above content is generated by AI and is for reference only.