The Integer Alibi: Localizing Cross-Kernel Divergence in INT8-Quantized LLM Inference
Swapping only the INT8 linear kernel (CUTLASS vs Triton) inside vLLM while holding all other factors fixed produces zero end-to-end sequence agreement across 1.7B and 8B Qwen3 models, despite each kernel being internally reproducible bit-for-bit The "integer alibi" proves the INT32 accumulator cannot be the divergence source: under verified no-overflow bounds, the dot product is exact and order-independent, localizing differences to post-accumulator scale application and output rounding Under po
Analysis
TL;DR
- Swapping only the INT8 linear kernel (CUTLASS vs Triton) inside vLLM while holding all other factors fixed produces zero end-to-end sequence agreement across 1.7B and 8B Qwen3 models, despite each kernel being internally reproducible bit-for-bit
- The "integer alibi" proves the INT32 accumulator cannot be the divergence source: under verified no-overflow bounds, the dot product is exact and order-independent, localizing differences to post-accumulator scale application and output rounding
- Under power-of-two scales, both kernels produce bit-identical outputs across all 196 layers (1.7B) and 252 layers (8B), with real-scale differences capped at one bfloat16 spacing; a probe checkpoint intervention fully restores end-to-end bitwise agreement
- FP8 GEMM exhibits a fundamentally different divergence signature: both prevalence and magnitude of differences scale with reduction depth, unlike INT8 where discrepancies remain at parts-per-million levels across a 64x K range
- Teacher-forced replay maps flips to tokens with small logit margins, achieving ROC-AUC 0.94 for flip-risk prediction across 16,384 positions, with full pre-registration, per-layer predictions, and a conformance procedure to be released
Why It Matters
This work directly challenges a foundational assumption in production LLM deployment—that interchangeable GPU kernels yield identical results—revealing that kernel choice alone can completely alter model outputs in quantized inference. For practitioners running INT8-quantized models, the findings imply that kernel provenance and version pinning are critical reproducibility concerns, not merely performance optimizations. The conformance procedure and released manifests offer the community a concrete methodology for auditing kernel interchangeability claims.
Technical Details
- Experimental design: Controlled swap of only the INT8 GEMM kernel (CUTLASS vs Triton) inside vLLM, with checkpoint, prompts, hardware, inference engine, decoding strategy, and quantization config held constant; tested on Qwen3-1.7B (196 linear layers) and Qwen3-8B (252 layers)
- Integer alibi proof: Demonstrated that for shared INT8 operands under a verified no-overflow bound, the INT32 dot product accumulator is exact and order-independent, mathematically ruling out the accumulator as a divergence source
- Layer-wise localization: Feeding identical operands from every linear layer to both kernels confirmed bit-identical outputs under power-of-two scales (196/196 and 252/252 pinned predictions), with real-scale deviations bounded to at most one bfloat16 ULP—localizing divergence strictly to scale application and output rounding after the accumulator
- FP8 contrast: Cross-implementation FP8 GEMM shows divergence magnitude and prevalence growing with reduction depth, while INT8 stays stable at ppm-level across a 64x range of K dimensions, indicating fundamentally different numerical behavior between quantization formats
- Flip prediction: Teacher-forced replay links layer-level flips to specific tokens; small logit margins are the primary predictor of flip risk, achieving ROC-AUC 0.94 on 16,384 positions; a probe checkpoint intervention restores full end-to-end bitwise agreement (8/8 and 16/16 sequences)
Industry Insight
- Kernel interchangeability cannot be assumed in production quantized inference pipelines; organizations should adopt kernel pinning and conformance testing as part of their deployment verification, especially for safety-critical or regulated applications
- The INT8 vs FP8 divergence signatures suggest that quantization format choice carries distinct reproducibility trade-offs: INT8 offers tighter numerical bounds but hidden rounding divergence, while FP8 exhibits depth-dependent error accumulation that may be more predictable but harder to bound
- The released conformance procedure and pre-registration framework establish a new standard for reproducibility auditing in ML systems, encouraging the community to treat kernel-level verification as a first-class concern alongside model-level reproducibility
Disclaimer: The above content is generated by AI and is for reference only.