Evaluating the Effect of Frame Rate in Sequence-Based Classification of Autism-Related Self-Stimulatory Hand Idiosyncrasies 评估帧率对基于序列分类的自闭症相关自我刺激手癖好识别的影响

TL;DR

• GRU and LSTM models significantly outperform CNN baselines in classifying autism-related self-stimulatory behaviors, achieving peak accuracies of 98.75% and 97.5% respectively.
• Optimal temporal sampling was identified at 15-frame intervals, demonstrating that downsampling can enhance performance compared to processing every frame.
• Data augmentation ablation studies revealed that while horizontal flips yield high standalone accuracy, upsampling is critical for maintaining performance in complex behavioral video tasks.
• Personalized machine learning approaches, involving per-subject model training on temporally split video segments, produced consistent predictions with low variance.

Why It Matters

This research provides critical empirical guidance for developing scalable, remote screening tools for Autism Spectrum Disorder (ASD), addressing the urgent need for computational methods in clinical settings where data is scarce. By establishing optimal architectures and sampling rates, it enables more efficient and accurate automated behavioral analysis, potentially reducing reliance on intensive manual screening processes. The findings also offer actionable insights into data augmentation strategies, helping practitioners maximize model performance even with limited datasets.

Technical Details

• Architectures: Compared Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) networks against prior Convolutional Neural Network (CNN) baselines using pose-derived features from the SSBD dataset.
• Temporal Sampling: Evaluated frame sampling intervals of 1, 5, 15, 30, 45, and 90 frames, identifying 15-frame intervals as optimal for both LSTM (97.5% accuracy) and GRU (98.75% accuracy).
• Data Augmentation: Conducted an ablation study on ten augmentation strategies within an I3D transfer learning pipeline, finding that excluding upsampling caused the largest performance drop, while horizontal flips achieved the highest standalone accuracy (48.78%).
• Personalized Modeling: Implemented a per-subject training approach on temporally split video segments, resulting in consistent predictions with a mean loss of 1.84 and standard deviation of 0.79.

Industry Insight

Practitioners should prioritize sequence-based models like GRUs or LSTMs over static CNNs for video-based behavioral classification tasks, particularly when leveraging pose estimation features. When working with limited clinical datasets, incorporating upsampling in data augmentation pipelines is essential to prevent significant performance degradation, even if other techniques like flipping show strong individual results. Finally, adopting personalized modeling strategies can improve consistency in predictions for individual subjects, offering a viable path toward robust, scalable diagnostic tools in data-scarce medical domains.

TL;DR

• 研究聚焦于利用视频序列分类自闭症相关的自我刺激手部行为，旨在解决远程筛查中可扩展计算方法的缺失问题。
• LSTM和GRU模型在姿态特征提取上表现优异，分别在每15帧采样间隔下达到97.5%和98.75%的准确率，显著优于之前的CNN基线（62-76%）。
• 针对小数据集的数据增强策略进行了消融实验，发现水平翻转效果最佳，而上采样对于复杂行为视频增强不可或缺。
• 个性化机器学习方法（按受试者训练模型）在时间分割的视频片段上产生了高度一致的预测结果，为数据稀缺的临床领域提供了具体指导。

为什么值得看

本文揭示了时序神经网络在处理小规模临床行为视频数据时的巨大潜力，证明了通过优化采样率和架构选择可以大幅提升诊断精度。同时，其关于数据增强的细致分析为资源受限的医疗AI应用提供了可复现的最佳实践指南。

技术解析

• 模型架构与性能对比：研究对比了LSTM和GRU两种循环神经网络架构，输入为来自SSBD数据集的姿态衍生特征。结果显示，两者均超越了传统的CNN基线，其中GRU在每15帧采样时达到最高准确率98.75%，LSTM为97.5%。
• 时间采样率优化：实验测试了1至90帧不等的采样间隔，发现并非采样越密集越好。每15帧的采样间隔在保留关键时序信息的同时有效降低了噪声和计算冗余，达到了性能峰值。
• 数据增强策略分析：在I3D迁移学习管道中评估了十种数据增强策略。水平翻转单独使用时准确率最高（48.78%），但消融研究表明移除上采样会导致性能大幅下降，证实了其在处理复杂行为视频时的必要性。
• 个性化建模验证：采用按受试者单独训练和测试的方法，对视频进行时间分割。该方法表现出极高的稳定性，平均损失为1.84，标准差仅为0.79，表明个性化模型能有效适应个体差异。

行业启示

• 临床AI落地需重视时序建模：在行为识别等动态任务中，基于RNN/LSTM的时序模型结合姿态特征可能比纯视觉CNN更具优势，尤其是在数据量有限的情况下。
• 数据增强策略应因地制宜：通用增强手段并不总是最优，必须通过消融实验确定特定领域（如医疗行为视频）的关键增强组件，避免盲目堆砌增强技术。
• 个性化模型是解决小样本问题的有效路径：在隐私敏感且数据稀缺的临床场景中，针对个体微调的个性化机器学习方案能提供比通用模型更稳定、可靠的预测结果。