KSE-Web: An Analysis of Hybrid Retrieval and LLM-Assisted Query Expansion for Low-Resource Khmer Semantic Search
KSE-Web introduces a new Khmer semantic search dataset with 3K cleaned full-text documents and 300 manually reviewed user-style queries with silver relevance labels Character n-gram BM25 achieves the strongest performance (0.943 Recall, 0.876 nDCG), outperforming multilingual dense retrieval alone Hybrid BM25+dense retrieval performs comparably to BM25 alone (0.929 Recall, 0.871 nDCG), suggesting dense models add limited value for low-resource Khmer LLM-assisted query expansion does not outperfo
Analysis
TL;DR
- KSE-Web introduces a new Khmer semantic search dataset with 3K cleaned full-text documents and 300 manually reviewed user-style queries with silver relevance labels
- Character n-gram BM25 achieves the strongest performance (0.943 Recall, 0.876 nDCG), outperforming multilingual dense retrieval alone
- Hybrid BM25+dense retrieval performs comparably to BM25 alone (0.929 Recall, 0.871 nDCG), suggesting dense models add limited value for low-resource Khmer
- LLM-assisted query expansion does not outperform baseline retrieval; however, Qwen2.5-3B significantly outperforms Qwen2.5-0.5B, indicating model scale matters for expansion quality
- Direct LLM expansion introduces topic drift, generic terms, and noisy reformulations, while simple filtering risks removing useful semantic cues
Why It Matters
This work addresses a critical gap in low-resource language NLP by providing one of the first annotated Khmer search datasets and systematically evaluating retrieval strategies for a language with unique challenges like ambiguous word boundaries and mixed Khmer-English usage. For AI practitioners working with underrepresented languages, it demonstrates that simpler retrieval methods can still dominate over complex LLM-based approaches, offering practical guidance on when to invest in expensive expansion techniques versus relying on robust baseline methods.
Technical Details
- Dataset constructed from ~17K candidate Khmer titles, filtered down to 3K cleaned full-text documents after normalization, deduplication, and document-length control; includes 300 manually reviewed user-style Khmer queries with silver relevance labels and partial human verification
- Evaluated four retrieval approaches: character n-gram BM25, multilingual dense retrieval, hybrid BM25+dense retrieval, and LLM-assisted query expansion using Qwen2.5 models (0.5B and 3B variants)
- BM25 achieved 0.943 Recall and 0.876 nDCG; hybrid retrieval achieved 0.929 Recall and 0.871 nDCG; dense retrieval alone performed notably lower
- LLM expansion analysis revealed that larger models (Qwen2.5-3B) produced substantially better expanded queries than smaller ones (Qwen2.5-0.5B), but neither matched non-expanded retrieval performance
- Identified key failure modes of LLM expansion: topic drift, introduction of generic terms, noisy reformulations, and the risk that simple filtering may discard useful semantic cues
Industry Insight
- For low-resource language search systems, investing in robust lexical retrieval (e.g., character n-gram BM25) should be prioritized over expensive LLM-based query expansion until language-specific embedding models and larger-scale annotated datasets become available
- When deploying LLM-assisted retrieval for underrepresented languages, model scale is a decisive factor—smaller models (sub-1B parameters) may produce insufficiently quality expansions, and careful filtering strategies are needed to avoid both topic drift and over-aggressive noise removal
- The KSE-Web dataset and findings provide a benchmark for future Khmer-aware retrieval model development, highlighting the need for language-specific embedding models and stronger human-verified annotations to advance the field
Disclaimer: The above content is generated by AI and is for reference only.