Exact and Certified Data Shapley for Weighted k-Nearest-Neighbor Regression and Soft-Label Prediction
Introduces the first pseudo-polynomial-time exact algorithm for weighted k-Nearest-Neighbor (KNN) regression Data Shapley, overcoming previous exponential complexity barriers. Provides a certified Fixed-Parameter Tractable Approximation Scheme (FPTAS) with machine-checkable error bounds for continuous weights and targets. Extends exact Data Shapley computation to weighted soft-label multi-class prediction, addressing a significant gap in existing toolkit capabilities. Releases an open-source, CP
Analysis
TL;DR
- Introduces the first pseudo-polynomial-time exact algorithm for weighted k-Nearest-Neighbor (KNN) regression Data Shapley, overcoming previous exponential complexity barriers.
- Provides a certified Fixed-Parameter Tractable Approximation Scheme (FPTAS) with machine-checkable error bounds for continuous weights and targets.
- Extends exact Data Shapley computation to weighted soft-label multi-class prediction, addressing a significant gap in existing toolkit capabilities.
- Releases an open-source, CPU-only library and establishes the first ground truth for weighted-regression Data Shapley, enabling rigorous auditing of stochastic estimators.
- Demonstrates that while Monte Carlo methods are statistically equivalent in mean value, they fail to reproduce exact rankings, highlighting the necessity of deterministic methods for auditability.
Why It Matters
This research resolves a long-standing computational bottleneck in Data Shapley estimation, specifically for weighted KNN regression, which was previously limited to inefficient brute-force methods. By providing exact algorithms and certified approximations, it enables reliable, deterministic auditing of training data influence, which is critical for regulatory compliance and model transparency in high-stakes applications.
Technical Details
- Algorithmic Innovation: Develops a counting dynamic program over the joint integer state space (sum of weights, sum of weighted targets) to achieve pseudo-polynomial time complexity, effectively handling the non-additive nature of weighted regression predictions.
- Certified Approximation: Implements an FPTAS that generates per-value error certificates, verified across 86,400 checks without violation, ensuring theoretical guarantees for continuous domains.
- Complexity Analysis: Establishes an unconditional $\Omega(D_w)$ output-size lower bound and provides access-model hardness results, defining the fundamental limits of the problem.
- Validation: Verified against exhaustive enumeration on 12,716 adversarial instances with zero mismatch, confirming the correctness of the exact algorithm.
- Extension: Generalizes the approach to handle soft-label multi-class prediction, broadening the applicability of exact Data Shapley beyond standard regression tasks.
Industry Insight
- Auditability Over Speed: For applications requiring strict accountability (e.g., healthcare, finance), deterministic exact methods should replace stochastic Monte Carlo estimators to ensure reproducible data valuation and ranking.
- Tooling Integration: The release of an open-source CPU-only library allows practitioners to integrate exact Data Shapley calculations into existing pipelines like pyDVL and OpenDataVal without requiring GPU acceleration.
- Benchmarking Standard: The provided ground truth serves as a new benchmark for evaluating the accuracy and convergence of approximate Data Shapley algorithms, driving future improvements in efficiency.
Disclaimer: The above content is generated by AI and is for reference only.