Modeling Is the Product
The "knowledge layer" in Data AI agents is often misidentified as merely the context layer (markdown, descriptions, glossary), when the modeling and metric layers underneath are what actually determine whether an agent works correctly Before AI agents, human analysts acted as both volume gates and quality filters; removing them exposes gaps in data modeling that were previously hidden The modeling layer (pre-joined, materialized tables) and metric layer (governed definitions) perform compute and
Analysis
TL;DR
- The "knowledge layer" in Data AI agents is often misidentified as merely the context layer (markdown, descriptions, glossary), when the modeling and metric layers underneath are what actually determine whether an agent works correctly
- Before AI agents, human analysts acted as both volume gates and quality filters; removing them exposes gaps in data modeling that were previously hidden
- The modeling layer (pre-joined, materialized tables) and metric layer (governed definitions) perform compute and arithmetic once, rather than per-query, ensuring consistency, accuracy, and cost efficiency
- An agent pointed at raw tables will guess business definitions (revenue, exclusions, filters) at query time, producing confidently formatted but potentially wildly incorrect answers
- The agent is an interface that reveals the quality of underlying data modeling; the "unglamorous" modeling work is the actual moat, not the chat interface
Why It Matters
This article challenges the prevailing industry narrative that investing in prompt engineering and context-layer tooling is sufficient for successful Data AI agents. For practitioners building or buying talk-to-your-data solutions, it serves as a critical reminder that without a governed modeling and metric layer, agents will produce fast, confident, but fundamentally unreliable answers at scale.
Technical Details
- Three distinct layers: The modeling layer (physical tables with pre-computed joins and materialized aggregates), the metric layer (governed business definitions like revenue, distinct counts, ratios, time windows), and the context layer (markdown, column descriptions, business glossary, routing hints)
- Compute-once vs. compute-per-query: The modeling layer resolves grain, joins, and aggregates ahead of time so agents read small tables instead of scanning large raw ones; the metric layer encodes business logic (exclusions, rate changes, fraud filters) that would otherwise be guessed at query time
- Variance problem: Without governed layers, the same question asked twice can take different query paths and return different numbers; with them, results are deterministic and identical across runs
- Auto-population as bootstrap only: Generating context from query logs, column lineage, and existing docs is a reasonable starting point but should not replace building the underlying modeling and metric layers
- Two routing paths: Metric questions (e.g., "revenue last month") route through the metric layer then the modeling layer; non-metric compositional questions (e.g., "which channels had the most declines on Thursday") bypass the metric layer and go directly to the modeling layer
Industry Insight
- Organizations should resist the temptation to treat the context layer as a plug-and-play setup step; investing in governed modeling and metric layers is a prerequisite, not an optional enhancement, for any production Data AI agent
- The agent itself is not a substitute for data infrastructure—it is a mirror that exposes whether your data was useful all along; companies with weak modeling layers will see their deficiencies amplified, not hidden
- Data teams should focus on accelerating the velocity of model creation in the governance layer rather than offloading definition work onto agents; the competitive moat remains the unglamorous years of modeling work, not the chat interface
Disclaimer: The above content is generated by AI and is for reference only.