NANQ: Noise-Floor-Aware Mixed-Precision Non-Uniform Quantization for Analog Compute-in-Memory
NANQ is a noise-aware mixed-precision non-uniform quantization framework designed specifically for analog compute-in-memory (CIM) hardware Unlike prior methods that minimize ideal quantization error, NANQ explicitly models magnitude-dependent weight noise from real eFlash CIM array measurements The framework converts hardware noise profiles into adaptive quantization density, allocating finer resolution to low-noise weight regions while avoiding wasted precision in noise-dominated regions Layer-
Analysis
TL;DR
- NANQ is a noise-aware mixed-precision non-uniform quantization framework designed specifically for analog compute-in-memory (CIM) hardware
- Unlike prior methods that minimize ideal quantization error, NANQ explicitly models magnitude-dependent weight noise from real eFlash CIM array measurements
- The framework converts hardware noise profiles into adaptive quantization density, allocating finer resolution to low-noise weight regions while avoiding wasted precision in noise-dominated regions
- Layer-wise bit-width assignment is determined by identifying each layer's precision saturation point under hardware noise using a unified threshold
- On-chip eFlash CIM SoC experiments show NANQ improves vision model accuracy by 8.05 percentage points and reduces language model PPL by 54.7% on average compared to PowerQuant under 2-bit weight-magnitude quantization, achieving most gains with only 3.2-3.8 equivalent bits
Why It Matters
This work addresses a critical gap in the deployment of quantized neural networks on analog compute-in-memory hardware, where device variation and read noise have historically undermined low-bit quantization accuracy. By incorporating real hardware noise characteristics directly into the quantization framework, NANQ enables practitioners to extract significantly more accuracy from extremely low-bit representations, making analog CIM a more viable path toward energy-efficient edge AI inference.
Technical Details
- NANQ models magnitude-dependent weight noise derived from measured responses of an eFlash CIM array, capturing the relationship between weight magnitude and hardware noise floor rather than assuming uniform or idealized noise distributions
- The noise profile is converted into an adaptive quantization density function that assigns non-uniform quantization levels, providing finer resolution in low-noise weight regions and coarser resolution where noise dominates
- A unified threshold-based method identifies each layer's precision saturation point under hardware noise, enabling automatic layer-wise bit-width assignment without per-layer manual tuning
- On-chip validation was performed on an eFlash CIM system-on-chip (SoC), demonstrating that mixed-precision NANQ captures most of the accuracy gains from additional quantization resources while maintaining an effective bit-width of only 3.2-3.8 bits
- Baseline comparison against PowerQuant showed substantial improvements: +8.05 percentage points in vision model accuracy and -54.7% average reduction in language model perplexity under 2-bit weight-magnitude quantization
Industry Insight
- Hardware-aware quantization frameworks that incorporate real measured noise profiles rather than idealized assumptions will become increasingly important as analog CIM architectures move from research prototypes to production deployment
- The finding that precision saturation points can be identified with a unified threshold across layers suggests a practical path toward automated, deployment-ready quantization pipelines that reduce the engineering overhead of mixed-precision model optimization
- For organizations investing in analog in-memory computing hardware, NANQ demonstrates that extreme low-bit quantization (2-bit) can be made viable through noise-aware algorithm-hardware co-design, potentially unlocking significant energy efficiency gains for edge AI applications
Disclaimer: The above content is generated by AI and is for reference only.