Five Ways to Compute an STFT on a Qualcomm SoC: Accuracy, Latency, and Power, Measured Not Guessed 在Qualcomm SoC上计算STFT的五种方法：准确性、延迟和功耗，测量而非猜测

TL;DR

• The article compares five implementations of Short-Time Fourier Transform (STFT) on a Qualcomm Snapdragon SM8650 SoC, evaluating accuracy, latency, and power consumption.
• CPU-based FFT libraries (PFFFT and PocketFFT) offer the lowest latency and highest accuracy but consume more power compared to other methods.
• Offloading STFT to the cDSP or NPU introduces higher latency due to cross-core communication overhead but can be beneficial when fused into larger pipelines.
• The QNN (Conv1d, fp16) path on the NPU has significantly higher error margins due to reduced precision but may still be acceptable depending on downstream application requirements.
• Power measurements reveal trade-offs between computational efficiency and energy usage across different backends.

Why It Matters

This analysis is crucial for AI practitioners working with audio processing tasks in edge devices, particularly those involving real-time applications like voice assistants or noise suppression systems. Understanding how different hardware components handle fundamental operations such as STFT helps optimize performance while managing resource constraints effectively. For researchers exploring heterogeneous computing architectures within mobile platforms, this study provides empirical evidence supporting strategic decisions about where specific computations should reside based on their unique characteristics and integration needs.

Technical Details

• Five distinct approaches were tested: two using general-purpose FFT libraries running directly on ARM cores (CPU PFFFT and CPU PocketFFT), one utilizing Qualcomm's High-Level Language (QHL) library designed specifically for digital signal processing tasks executed via compute DSP (cDSP), another custom-built solution also targeting cDSP aimed at optimizing multi-stage workflows through fusion techniques, and finally leveraging neural network processors by converting Discrete Fourier Transforms into matrix multiplications compatible with low-precision arithmetic supported by TensorRT-style frameworks operating under HTP-NPU environment.
• All implementations processed identical input parameters including chunk size (512 samples), left context length (64 samples), padding strategy (right-side reflection), FFT point count (256 points), hop window value (128 samples), number of output frames per chunk (four total), magnitude bin resolution (129 bins). Additionally, they utilized consistent test signals derived from standard WAV files sampled at 16 kHz rate ensuring fair comparison conditions throughout experiments conducted remotely yet synchronously over SSH tunnels connecting local development environments against physical test units equipped with necessary instrumentation setups capable of capturing precise temporal dynamics alongside electrical characteristics during runtime phases.
• Performance metrics collected encompassed three primary dimensions namely numerical fidelity quantified via maximum absolute deviation relative against double-precision reference outputs generated using NumPy library functions; response times measured individually per segment then averaged across thousands iterations yielding representative figures indicative of typical operational behavior; lastly power draw monitored continuously employing specialized hardware interfaces interfaced programmatically enabling granular tracking without disrupting normal execution flows thereby providing actionable insights regarding overall system health implications associated with various deployment scenarios.

Industry Insight

When designing embedded solutions requiring efficient spectral feature extraction prior feeding subsequent machine learning models engineers must carefully weigh multiple factors simultaneously rather than focusing solely on raw speed improvements alone since suboptimal choices could lead unintended consequences elsewhere down line potentially negating initial gains achieved elsewhere elsewhere entirely! Therefore adopting holistic evaluation methodologies incorporating diverse perspectives becomes essential especially considering increasingly complex interdependencies emerging among subsystems comprising modern autonomous vehicles equipped advanced sensory suites relying heavily upon sophisticated algorithms capable interpreting vast amounts data acquired rapidly changing surroundings encountered daily basis journeys undertaken passengers alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike alike

TL;DR

• 在骁龙SM8650上对比了五种STFT实现路径（CPU、cDSP、HTP-NPU），发现无绝对“最佳”方案，需根据下游任务权衡精度、延迟与功耗。
• CPU通用FFT库（PFFFT/PocketFFT）在延迟和精度上表现最优，但NPU路径因fp16量化导致误差高一个数量级，适合对精度不敏感的场景。
• cDSP与NPU的延迟劣势源于跨核通信开销，而非计算本身慢；仅当算子融合或批量处理时才能摊薄成本，体现异构计算中“调度成本大于计算成本”的工程现实。

为什么值得看

本文揭示了边缘音频AI管线中最常被忽视却至关重要的前置模块——STFT的实际工程约束，强调不能脱离整体架构孤立评估单个算子的性能。对于车载语音助手、舱内感知等实时性敏感的应用，该研究提供了可复现的测量框架与决策依据，帮助工程师避免盲目追求“硬件加速”而陷入精度或延迟陷阱。

技术解析

所有五路实现均基于相同数学定义：512样本输入块、64样本左上下文、反射填充、256点FFT、128样本步长、Hann窗、每块输出4帧、每帧129个幅度 bin，使用16kHz WAV信号测试。

• CPU PFFFT & PocketFFT：分别使用双精度与单精度通用FFT库，在NumPy参考下几乎零误差，微秒级延迟，是速度与精度的基准。
• cDSP (QHL)：高通官方DSP库，同样保持高精度，但因FastRPC跨进程调用引入显著延迟（约1ms+），不适合独立小任务。
• cDSP (custom)：专为流水线融合设计，虽单独FFT不占优，但在多阶段算子合并后可有效摊薄通信开销，体现“系统级优化优于单点加速”。
• HTP-NPU (QNN, Conv1d, fp16)：将DFT重写为矩阵乘法运行于NPU，支持端到端图优化，但fp16导致最大误差达0.018，且延迟受限于QNN runtime调度，仅适用于对精度容忍度高且需与其他模型层紧密集成的场景。

行业启示

• 边缘设备上的AI部署不应以单一算子性能论英雄，必须结合完整pipeline进行联合建模与资源调度，尤其注意通信/同步开销在小任务中的主导作用。
• NPU加速并非万能解药，其价值在于大规模并行与算子融合能力，对于轻量级预处理步骤如STFT，传统CPU方案可能更经济高效；开发者应建立“按需选择后端”的评估体系而非默认迁移至加速器。
• 随着智能座舱、可穿戴设备等场景对低功耗连续音频处理需求增长，未来SoC厂商需在硬件层面降低跨域通信延迟（如改进FastRPC或引入专用信号处理引擎），同时提供标准化的混合精度算子库以平衡效率与准确性。