OpenSearch Optimizations for Production RAG, Part 2: Lexical Retrieval
Lexical retrieval in OpenSearch complements semantic vector search by excelling at exact identifiers, proper nouns, and typo-tolerant partial matches where semantic models often fail. RAG corpora are categorized into two distinct shapes: unstructured document corpora (prose-heavy) and structured corpora (multi-field with metadata), requiring different indexing and querying strategies. Controlled values (enums) should utilize `keyword` fields with `term` or `terms` queries for exact, non-scored,
Analysis
TL;DR
- Lexical retrieval in OpenSearch complements semantic vector search by excelling at exact identifiers, proper nouns, and typo-tolerant partial matches where semantic models often fail.
- RAG corpora are categorized into two distinct shapes: unstructured document corpora (prose-heavy) and structured corpora (multi-field with metadata), requiring different indexing and querying strategies.
- Controlled values (enums) should utilize
keywordfields withtermortermsqueries for exact, non-scored, cache-friendly filtering, avoiding analyzers entirely. - Free-text fields rely on BM25 as the default scoring model, which balances term frequency, inverse document frequency, and length normalization to determine relevance.
- Production RAG systems should implement a hybrid approach, running both semantic and lexical paths simultaneously to maximize recall and coverage across diverse query types.
Why It Matters
This article provides critical architectural guidance for building robust Retrieval Augmented Generation (RAG) systems, emphasizing that semantic search alone is insufficient for production environments. By detailing how to effectively leverage lexical retrieval alongside vector search, it helps engineers avoid common pitfalls such as missing exact entity matches or misusing scoring mechanisms on controlled data. Understanding these distinctions allows practitioners to optimize for both high recall and low latency, ensuring LLMs receive the most accurate and relevant context.
Technical Details
- Corpus Shapes: The article distinguishes between "document corpora" (single blob of text, e.g., articles, tickets) and "structured corpora" (multiple typed fields, e.g., job postings with title, location, skills). Structured corpora require complex field weighting and metadata filtering, while document corpora focus on body text scoring.
- Controlled Value Handling: For fields with closed sets of values (e.g., country codes, status flags), use
keywordmappings instead oftext. Query these usingtermortermswithin aboolfilter context to ensure exact matching without scoring overhead or tokenization issues. - BM25 Scoring Mechanics: BM25 is the default similarity model for free-text fields. It calculates relevance based on three components: Term Frequency (diminishing returns for repeated terms), Inverse Document Frequency (higher weight for rarer terms), and Length Normalization (penalizing longer documents for accidental term inclusion). Tuning parameters
k1andbcontrol saturation and normalization strength. - Query Construction: Minimal BM25 queries target specific text fields. For structured data, queries must combine text matching (
match/multi_match) with metadata filters to drive both scoring and filtering logic appropriately.
Industry Insight
- Adopt Hybrid Search Architectures: Do not rely solely on vector embeddings. Implement a dual-path retrieval system that combines semantic search for intent/synonymy with lexical search for precision/exactness to cover edge cases like proper nouns and rare terms.
- Strict Data Typing for Metadata: Enforce rigorous schema design by separating controlled categorical data from free-text fields. Using
keywordtypes for enums prevents unnecessary computational overhead and ensures predictable, cache-efficient filtering behavior. - Optimize for Corpus Structure: Tailor indexing strategies based on whether the data is primarily unstructured prose or highly structured records. Structured data benefits significantly from field-level boosting and precise metadata filtering, whereas unstructured data requires careful attention to analyzer chains and BM25 parameter tuning.
Disclaimer: The above content is generated by AI and is for reference only.