WHERE to Generate Matters: Budget-Aware Synthetic Augmentation for Label Skewed Federated Learning
FedEAS introduces a budget-aware synthetic data augmentation strategy for Federated Learning to address label skew without prohibitive computational costs. The method uses entropy-adaptive per-class generation budgets derived from local label distributions to determine both the quantity and class distribution of synthetic samples. FedEAS reduces the total generation budget by 94.1% compared to full class balancing while recovering most of the associated accuracy gains. Experiments on CIFAR-10 an
Analysis
TL;DR
- FedEAS introduces a budget-aware synthetic data augmentation strategy for Federated Learning to address label skew without prohibitive computational costs.
- The method uses entropy-adaptive per-class generation budgets derived from local label distributions to determine both the quantity and class distribution of synthetic samples.
- FedEAS reduces the total generation budget by 94.1% compared to full class balancing while recovering most of the associated accuracy gains.
- Experiments on CIFAR-10 and CIFAR-100 demonstrate that FedEAS outperforms uniform allocation strategies by up to 18.82% under equivalent budget constraints.
Why It Matters
This research addresses a critical bottleneck in practical Federated Learning: the high computational overhead of mitigating non-IID data distributions. By decoupling performance gains from massive synthetic data generation, FedEAS makes label-skew correction feasible for resource-constrained edge devices and large-scale distributed systems.
Technical Details
- FedEAS Policy: A novel framework that calculates an entropy-adaptive budget for each client based on their local label distribution, dynamically allocating resources to underrepresented classes.
- Budget Mechanism: Unlike fixed-budget approaches, the total generation budget is emergent, determined by the sum of per-client, per-class allocations, allowing for efficient resource usage.
- Performance Metrics: The approach achieves a 94.1% reduction in generation budget relative to full balancing methods and shows up to 18.82% accuracy improvement over uniform allocation on standard vision benchmarks.
- Problem Context: Specifically targets client drift and global accuracy degradation caused by label skew in FL settings using synthetic data augmentation.
Industry Insight
- Resource Optimization: Organizations deploying FL should prioritize adaptive, entropy-based augmentation strategies over uniform or full-balancing approaches to significantly lower inference and training costs.
- Scalability: This method enables the scaling of FL systems to larger numbers of heterogeneous clients by reducing the communication and compute burden associated with data balancing.
- Implementation Focus: Developers should focus on integrating local label distribution analysis into the client-side preprocessing pipeline to enable dynamic budget allocation before model aggregation.
Disclaimer: The above content is generated by AI and is for reference only.