Run Massive-Scale UMAP in Minutes Using Multiple GPUs—Without Losing Accuracy
NVIDIA cuML and cuVS 25.06 introduce multi-GPU support for the all-neighbors kNN graph construction step in UMAP, removing the previous single-GPU training bottleneck The approach partitions datasets into balanced clusters with overlapping vectors across cluster boundaries, computes local kNN graphs independently per GPU, and merges them into a global graph without expensive all-to-all communication Benchmarks on MIRACL and Wiki datasets show up to 74x speedup over projected CPU runtimes using e
Analysis
TL;DR
- NVIDIA cuML and cuVS 25.06 introduce multi-GPU support for the all-neighbors kNN graph construction step in UMAP, removing the previous single-GPU training bottleneck
- The approach partitions datasets into balanced clusters with overlapping vectors across cluster boundaries, computes local kNN graphs independently per GPU, and merges them into a global graph without expensive all-to-all communication
- Benchmarks on MIRACL and Wiki datasets show up to 74x speedup over projected CPU runtimes using eight NVIDIA H100 GPUs, processing over 870 GB of vectors in 8 minutes
- Two key hyperparameters—knn_n_clusters and knn_overlap_factor—provide a controllable trade-off between memory usage, computation time, and embedding quality
- This makes UMAP feasible for multi-hundred-gigabyte workloads in minutes instead of hours or days, enabling iterative and exploratory analysis at massive scale
Why It Matters
This breakthrough directly addresses one of the most significant bottlenecks in large-scale dimensionality reduction: the all-neighbors kNN graph construction step that previously limited UMAP to single-GPU training regardless of dataset size. For AI practitioners working with embedding datasets, vector databases, or large-scale feature spaces, this means interactive exploration and parameter tuning become practically feasible at scales that were previously prohibitively expensive. The preservation of embedding quality at massive scale ensures that the speed gains do not come at the cost of analytical fidelity.
Technical Details
- Multi-GPU kNN graph construction: The dataset is partitioned into approximately balanced clusters using a k-means-based approach. Each GPU independently gathers data for its assigned clusters from CPU memory, computes local all-neighbors kNN graphs, and merges them into a global graph, eliminating the need for all-to-all communication that typically limits distributed kNN scalability.
- Out-of-core design with overlap: Vectors near cluster boundaries are assigned to multiple nearby clusters via the
knn_overlap_factorparameter, preserving nearest-neighbor relationships across partitions. This ensures the merged global graph maintains accuracy comparable to a single-GPU computation on the full dataset. - Key hyperparameters:
knn_n_clusterscontrols the number of data partitions (higher values reduce per-GPU memory but increase merge overhead), whileknn_overlap_factorcontrols how many closest clusters each point belongs to (higher values improve embedding quality at the cost of increased computation and memory). - Benchmark performance: On the MIRACL and Wiki datasets totaling over 870 GB of vectors, eight NVIDIA H100 GPUs achieved up to 74x speedup over projected CPU runtimes, completing UMAP in approximately 8 minutes while maintaining high trustworthiness scores in the resulting embeddings.
- API integration: The distributed all-neighbors graph construction is exposed through the cuVS all-neighbors API, with cuML UMAP using these parameters internally, ensuring backward compatibility with existing single-GPU workflows.
Industry Insight
- The elimination of the single-GPU training bottleneck for UMAP opens the door for real-time exploratory analysis on billion-vector datasets, which is increasingly common in retrieval-augmented generation (RAG) systems, vector database management, and large-scale embedding visualization pipelines.
- The explicit space-time-quality trade-off exposed through
knn_n_clustersandknn_overlap_factorgives practitioners fine-grained control, enabling them to optimize for interactive exploration (prioritizing speed) or production-grade analysis (prioritizing embedding fidelity) depending on their workflow stage. - As datasets continue to grow beyond single-GPU memory capacities, this out-of-core multi-GPU pattern—partition, overlap, compute locally, merge globally—serves as a reusable architectural template for other graph-based and nearest-neighbor-dependent algorithms beyond UMAP.
Disclaimer: The above content is generated by AI and is for reference only.