Padamitra: Grounded Glossary Generation for Classical Sanskrit
Introduces "grounded glossary generation," a novel NLP task that formalizes the traditional Sanskrit patha commentary practice into a structured, evaluable objective requiring models to recover semantically meaningful phrases and produce translation-grounded meanings from sloka-translation pairs Constructs a large-scale benchmark of 31,316 sloka-translation-glossary triples drawn from the Valmiki Ramayana and Srimad Bhagavatam, paired with two evaluation metrics: Jaccard similarity for phrase re
Analysis
TL;DR
- Introduces "grounded glossary generation," a novel NLP task that formalizes the traditional Sanskrit patha commentary practice into a structured, evaluable objective requiring models to recover semantically meaningful phrases and produce translation-grounded meanings from sloka-translation pairs
- Constructs a large-scale benchmark of 31,316 sloka-translation-glossary triples drawn from the Valmiki Ramayana and Srimad Bhagavatam, paired with two evaluation metrics: Jaccard similarity for phrase recovery and Meaning Faithfulness for semantic consistency
- Evaluates multiple modern LLMs (Gemma-3n-E4B, Gemma-3-12B, Phi-4, Qwen3.5-9B) across zero-shot, few-shot, and instruction fine-tuned settings, finding that instruction fine-tuning substantially outperforms prompting strategies
- Identifies over-segmentation of sandhi and samasa compounds as the dominant failure mode, highlighting morphological modeling as the key bottleneck for faithful Sanskrit lexical decomposition
Why It Matters
This work bridges classical Sanskrit scholarship and modern NLP by formalizing a traditional commentary practice into a rigorous benchmark, enabling systematic evaluation of LLMs on low-resource, morphologically rich languages. It demonstrates that even state-of-the-art models struggle with complex morphological phenomena like sandhi and samasa, pointing to a critical gap in current architectures for handling agglutinative and compounding languages.
Technical Details
- Task formulation: Grounded glossary generation requires models to decompose Sanskrit slokas into meaningful lexical units and assign translation-grounded meanings, mirroring the patha commentary tradition used by classical Sanskrit scholars
- Benchmark construction: 31,316 triples from two major Sanskrit texts (Valmiki Ramayana and Srimad Bhagavatam), with evaluation via Jaccard score for phrase-level recovery accuracy and Meaning Faithfulness metric for semantic consistency against reference glossaries
- Model evaluation: Tested Gemma-3n-E4B, Gemma-3-12B, Phi-4, and Qwen3.5-9B across three settings—zero-shot prompting, few-shot prompting, and instruction fine-tuning—with explicit segmentation strategies yielding additional performance gains
- Error analysis: Over-segmentation of sandhi (phonological concatenation) and samasa (compound) forms is the primary failure mode, indicating that current LLMs lack robust morphological decomposition capabilities for highly inflected languages
Industry Insight
- Morphological modeling remains a critical bottleneck for deploying LLMs on low-resource, morphologically rich languages; investing in explicit morphological preprocessing or specialized tokenization could unlock significant performance gains
- Instruction fine-tuning on domain-specific, culturally grounded tasks substantially outperforms general prompting, suggesting that targeted fine-tuning on classical text corpora is a viable strategy for building specialized language models
- The formalization of traditional scholarly practices (like patha commentary) into evaluable NLP benchmarks offers a replicable template for bringing other heritage knowledge systems into modern AI evaluation frameworks
Disclaimer: The above content is generated by AI and is for reference only.