GROUND: Reducing Hallucinations in LLM-Based Enterprise Analytics Through Governed Semantic Definitions
GROUND (Governed Retrieval Over Unified Normalized Definitions) is a framework that constrains LLM-generated analytics to a governed semantic layer, addressing hallucinations in enterprise text-to-SQL systems. The system validates generated SQL against six rule categories—schema, metric, join, grain, filter, and row-level security—before execution, retrying or abstaining on violations. In a 100-question synthetic benchmark, GROUND was the only system free of hallucinations across all six evaluat
Analysis
TL;DR
- GROUND (Governed Retrieval Over Unified Normalized Definitions) is a framework that constrains LLM-generated analytics to a governed semantic layer, addressing hallucinations in enterprise text-to-SQL systems.
- The system validates generated SQL against six rule categories—schema, metric, join, grain, filter, and row-level security—before execution, retrying or abstaining on violations.
- In a 100-question synthetic benchmark, GROUND was the only system free of hallucinations across all six evaluated categories, while ungoverned systems violated row-level security.
- Semantic-only grounding (exact metric definitions without access policies) still leaked data, proving that governance cannot be replaced by metric fidelity alone.
- Zero violations were observed on enforced guarantees (filters and row-level security) across four models from three providers, though judgment-dependent behaviors like refusing undefined metrics remain fallible.
Why It Matters
This work directly addresses the primary barrier to production deployment of LLM-based analytics: hallucinated or unsafe SQL generation. For AI practitioners building enterprise analytics tools, GROUND demonstrates that semantic governance and access control must be treated as first-class constraints, not afterthoughts. The findings challenge the common assumption that RAG over documentation or semantic layer definitions alone are sufficient for safe enterprise deployment.
Technical Details
- GROUND introduces a governed semantic layer that supplies approved metric definitions, dimensions, join paths, filters, and row-level security policies, binding user intent to these governed constructs before SQL generation.
- A multi-rule validation engine checks generated SQL against schema correctness, metric fidelity, valid join paths, correct grain, filter compliance, row-level security enforcement, and cost rules, with retry or abstention on violations.
- Evaluation includes a 100-question synthetic enterprise-reporting benchmark, replication on real U.S. NHTSA vehicle-safety data with hand-authored gold standards, and adversarial testing across four models from three providers.
- Comparative baselines include direct schema-only text-to-SQL, schema-RAG, and semantic-only grounding, all under a shared model to isolate the effect of governance.
Industry Insight
- Enterprise AI teams should invest in governed semantic layers with explicit access policies rather than relying solely on schema grounding or RAG; metric accuracy without security governance is insufficient.
- The zero-violation result on enforced guarantees across diverse models suggests that pre-execution validation frameworks can make LLM-based analytics production-ready, but organizations should still account for fallibility in judgment-dependent behaviors like metric refusal.
- The adversarial testing results highlight the need for robust security validation in any text-to-SQL pipeline, as even semantically correct systems can leak data without explicit row-level security enforcement.
Disclaimer: The above content is generated by AI and is for reference only.