Cross-Dialect Generalization Without Retraining: Benchmarks and Evaluation of Schema-Derived Constrained Decoding for MLIR
Introduces schema-derived constrained decoding for MLIR, allowing small language models to generate valid code without fine-tuning by using mechanical priors from Operation Definition Specifications. Releases four new benchmarks (MLIR-Spec-150, Linalg-Spec-30, StableHLO-Spec-30, StableHLO-Held-Out-200) totaling 410 natural-language-to-MLIR pairs, addressing the scarcity of MLIR data in pretraining corpora. Demonstrates that SmolLM2-1.7B matches or exceeds larger 15B-34B code LMs on structural di
Analysis
TL;DR
- Introduces schema-derived constrained decoding for MLIR, allowing small language models to generate valid code without fine-tuning by using mechanical priors from Operation Definition Specifications.
- Releases four new benchmarks (MLIR-Spec-150, Linalg-Spec-30, StableHLO-Spec-30, StableHLO-Held-Out-200) totaling 410 natural-language-to-MLIR pairs, addressing the scarcity of MLIR data in pretraining corpora.
- Demonstrates that SmolLM2-1.7B matches or exceeds larger 15B-34B code LMs on structural dialects like Linalg, achieving 80% verification validity at 8-25x faster generation speeds.
- Proposes a three-layer constraint stack (CFG, type-domain splits, SSA-scope validator) that generalizes across dialects without requiring new code for porting, such as moving from arith+func+memref+linalg to StableHLO.
Why It Matters
This research addresses a critical bottleneck in ML compiler infrastructure: the lack of high-quality MLIR data in code LLM training sets and the impracticality of fine-tuning separate models for every new dialect. By shifting adaptation from training time to inference time via schema constraints, it enables scalable, zero-shot cross-dialect generalization. This approach significantly reduces computational costs while improving code validity, making it highly relevant for developers building ML compilers and tools that rely on intermediate representations.
Technical Details
- Constraint Stack Architecture: Implements a three-layer system: (C1) Context-Free Grammar (CFG) over operation signatures, (C2) Type-domain splits extracted from an ODS-extracted type lattice, and (C3) an SSA-scope validator that drives a five-retry rejection sampling mechanism.
- Benchmark Suite: Provides Apache-2.0 licensed datasets including MLIR-Spec-150, Linalg-Spec-30, StableHLO-Spec-30, and StableHLO-Held-Out-200, along with out-of-grammar stress tests and functional reference sets, accompanied by Gebru datasheets and Croissant 1.0 metadata.
- Performance Metrics: On the Linalg dialect, SmolLM2-1.7B achieved an 80.0% verify-valid rate (mean over three seeds, n=125), outperforming CodeLlama-34B, Granite-Code-34B, and StarCoder2-15B by 21-44 percentage points.
- Generalization Capability: The method requires no new constraint-layer code to port between dialects, successfully applying constraints derived from arith+func+memref+linalg to StableHLO without modification.
- Limitations: The approach is less effective on dialects where verifier semantics depend on specific attribute values rather than structural constraints, such as arith+func and templated parametric StableHLO, where larger baselines performed better.
Industry Insight
- Cost-Efficient Model Deployment: Organizations can leverage smaller, faster models for complex compiler tasks by integrating schema constraints, reducing inference latency and hardware requirements compared to deploying massive 30B+ parameter models.
- Scalable Compiler Tooling: As MLIR dialects proliferate across different hardware accelerators and frameworks, this zero-shot generalization strategy offers a sustainable path for maintaining code generation tools without the overhead of continuous fine-tuning for each new domain.
- Data Scarcity Mitigation: For domains with limited training data like specialized IRs, combining lightweight models with rigorous syntactic/semantic constraints at inference time provides a viable alternative to data-hungry deep learning approaches.
Disclaimer: The above content is generated by AI and is for reference only.