Imbalanced Data Clustering via Targeted Data Augmentation Using GMM and LLM
A novel unsupervised data augmentation method combining Gaussian Mixture Models (GMMs) and Large Language Models (LLMs) to address imbalanced text data in clustering tasks GMMs are leveraged to detect clusters corresponding to underrepresented topics, while LLMs generate synthetic documents to enrich these minority clusters Experiments on multiple imbalanced text datasets show the approach preserves clustering performance across all cases and often improves cluster interpretability The method of
Analysis
TL;DR
- A novel unsupervised data augmentation method combining Gaussian Mixture Models (GMMs) and Large Language Models (LLMs) to address imbalanced text data in clustering tasks
- GMMs are leveraged to detect clusters corresponding to underrepresented topics, while LLMs generate synthetic documents to enrich these minority clusters
- Experiments on multiple imbalanced text datasets show the approach preserves clustering performance across all cases and often improves cluster interpretability
- The method offers a scalable solution for improving data representation in unsupervised NLP tasks without requiring labeled data
- Published at the 23rd International Symposium on Intelligent Data Analysis (IDA 2025), arXiv:2607.28635
Why It Matters
This work addresses a persistent and practically significant challenge in NLP: minority topic representation in unsupervised clustering, where traditional methods tend to overlook underrepresented classes. By combining probabilistic modeling (GMM) with generative LLM capabilities, it provides a label-free augmentation strategy that practitioners can deploy without expensive annotation pipelines. This is particularly relevant for real-world applications like document classification, topic modeling, and content organization where class imbalance is the norm rather than the exception.
Technical Details
- GMM-based cluster detection: Gaussian Mixture Models are used to identify and isolate clusters corresponding to underrepresented topics in the data, leveraging their flexibility and robustness in modeling multi-modal distributions without supervision.
- LLM-driven synthetic generation: Once minority clusters are identified, Large Language Models generate synthetic documents tailored to enrich these underrepresented areas, effectively balancing the dataset for downstream clustering.
- Unsupervised pipeline: The entire approach operates without labeled data, making it applicable to scenarios where ground-truth annotations are unavailable or prohibitively expensive to obtain.
- Benchmark evaluation: The method was tested on various imbalanced text datasets, with results demonstrating preserved clustering performance and enhanced interpretability of cluster outputs.
- Publication venue: Advances in Intelligent Data Analysis, 23rd International Symposium on Intelligent Data Analysis (IDA 2025), proceedings pp. 246-260.
Industry Insight
- Organizations dealing with real-world text data should consider hybrid GMM-LLM augmentation pipelines as a cost-effective alternative to manual data collection or supervised balancing techniques, especially in domains with scarce labeled data.
- The preservation of clustering performance alongside improved interpretability suggests this approach could unlock better topic discovery in enterprise document repositories, customer feedback analysis, and news categorization systems.
- As LLM costs continue to decrease and generation quality improves, targeted augmentation strategies like this will likely become a standard preprocessing step for unsupervised NLP pipelines, particularly in low-resource language and niche domain applications.
Disclaimer: The above content is generated by AI and is for reference only.