Prof-K: Probabilistic One-Pass Filtering for Efficient Top-k Selection
Prof-K is a fast, scalable, distribution-agnostic top-k selection algorithm that uses a single-pass probabilistic filtering procedure with formal correctness guarantees The method works by taking a small random sample to estimate an adaptive threshold, streaming N input elements once into a compact buffer, then running exact top-k on the buffer to recover true top-k elements with probability at least 1 - ε Theoretical guarantees are derived for both correctness and buffer size, along with an app
Analysis
TL;DR
- Prof-K is a fast, scalable, distribution-agnostic top-k selection algorithm that uses a single-pass probabilistic filtering procedure with formal correctness guarantees
- The method works by taking a small random sample to estimate an adaptive threshold, streaming N input elements once into a compact buffer, then running exact top-k on the buffer to recover true top-k elements with probability at least 1 - ε
- Theoretical guarantees are derived for both correctness and buffer size, along with an approximately optimal sample size that minimizes overhead as a function of N and k
- Empirically, Prof-K achieves 1.5x-10x speedups over PyTorch topk and RadiK, with the largest gains in the large-scale, small-to-moderate-k regime
- The algorithm provides a principled accuracy-speed trade-off by allowing relaxed recall targets (e.g., 95% recall) and is demonstrated on training BatchTopK Sparse Autoencoders where top-k selection is a significant training cost
Why It Matters
Top-k selection is a fundamental computational primitive used across databases, information retrieval, signal processing, and modern ML workloads like sparse activations and attention pruning. As data scales grow, existing exact methods become prohibitively expensive while approximate methods suffer from brittleness under adversarial or heavy-tailed inputs—Prof-K addresses this gap with distribution-agnostic probabilistic guarantees that ensure robustness where prior approaches fail.
Technical Details
- Single-pass filtering architecture: Prof-K estimates an adaptive threshold via random sampling, streams all N elements once into a compact buffer, and applies exact top-k on the buffered subset, achieving probabilistic correctness with probability ≥ 1 - ε
- Theoretical guarantees: High-probability bounds are derived for both correctness and buffer size, with an approximately optimal sample size formula that minimizes overhead as a function of input size N and target rank k
- Distribution-agnostic robustness: Unlike heuristic-based approximate methods, Prof-K's guarantees hold independently of input distribution, ensuring reliability under adversarial conditions and heavy-tailed data
- Accuracy-speed trade-off: Users can relax recall targets (e.g., recovering 95% of true top-k values) to achieve additional speedups while maintaining principled control over approximation quality
- Application to Sparse Autoencoders: Demonstrated impact on training BatchTopK SAEs, where top-k selection constitutes a significant portion of training computational cost
Industry Insight
- The 1.5x-10x speedups over highly optimized baselines like PyTorch topk suggest Prof-K could become a drop-in replacement for top-k operations in large-scale ML pipelines, particularly in attention mechanisms and sparse activation models where k is small relative to N
- The distribution-agnostic guarantees address a critical reliability concern in production systems where input distributions may be adversarial or unpredictable, making this especially valuable for safety-critical or deployed ML applications
- The demonstrated application to Sparse Autoencoders signals growing industry interest in interpretability tools—Prof-K's efficiency gains could accelerate SAE adoption and enable training on larger datasets that were previously computationally prohibitive
Disclaimer: The above content is generated by AI and is for reference only.