Where AI Value is Won and Lost
Enterprise AI underdelivery stems not from technology gaps but from misallocated domain expertise—most effort concentrates on deployment where ceilings are low, while critical framing and judgment phases are starved of expert input AI collapsed the "generation" phase of problem-solving from weeks to hours, but framing (intent, ontology, architecture) and judgment remain entirely dependent on domain knowledge that cheap generation cannot substitute Three framing pillars—Intent, Ontology, and Arch
Analysis
TL;DR
- Enterprise AI underdelivery stems not from technology gaps but from misallocated domain expertise—most effort concentrates on deployment where ceilings are low, while critical framing and judgment phases are starved of expert input
- AI collapsed the "generation" phase of problem-solving from weeks to hours, but framing (intent, ontology, architecture) and judgment remain entirely dependent on domain knowledge that cheap generation cannot substitute
- Three framing pillars—Intent, Ontology, and Architecture—form the foundation of any custom AI build, with ontology and architecture being the domain-bound elements non-experts cannot reliably settle
- A case study of a grid-connected battery scheduling model revealed three cascading framing failures (incorrect reserve market assumptions, collapsed ontology categories, impossible architecture) that produced no errors or warnings—only a domain expert's challenge could surface them
- Across 63 defects in 9 builds, only 2 were caught by automated checking; 35 were surfaced by domain challenge and 23 by adversarial review designed by domain-aware reviewers
Why It Matters
This article reframes the enterprise AI value problem from a technology bottleneck to a talent allocation decision, offering a practical framework for where domain experts should be positioned in AI builds. For practitioners, it explains the persistent gap between pilot activity and measurable ROI while providing an actionable blueprint for restructuring teams around the two high-leverage phases—framing and judgment—rather than the now-commoditized generation phase.
Technical Details
- Framing as three iterative pillars: Intent (what decision the output serves, who acts on it, boundary/horizon/resolution), Ontology (what entities exist and how they relate—e.g., product hierarchies, reserve product categorization), and Architecture (objective function design, constraint placement, model autonomy boundaries)
- Grid battery case study: A scheduling model for a grid-connected battery failed on all three pillars—incorrectly assuming full reserve capacity clearance (intent), collapsing two distinct reserve products into one category with identical delivery duration (ontology), and selling the same capacity block into three reserve products simultaneously (architecture), with zero automated detection
- Defect analysis across 63 logged defects in 9 builds: 35 surfaced by domain challenge, 23 by adversarial review from domain-aware reviewers, 3 by benchmark comparison, and only 2 by automated checking—demonstrating that models cannot check their own premises
- Benchmark degradation at scale: Industry-standard optimization tests show strong performance only because problems are pre-framed by humans; performance degrades sharply when models must handle unassisted framing at industrial scale
- Accelerated domain acquisition: AI can compress learning curves by building structured courses that map new domain concepts onto existing functional anchors, enabling experts to operate effectively in adjacent domains in days rather than weeks
Industry Insight
- Organizations should restructure AI teams to place domain experts at the framing and judgment boundaries—before any code is written and as independent verification—rather than embedding them in the now-automated generation loop, directly addressing the 95% pilot failure rate identified by MIT's NANDA initiative
- Investment in "scaffolded domain acquisition"—using AI to accelerate expertise in adjacent domains through structural analogy—should be treated as a strategic capability, enabling faster cross-domain builds without requiring lifelong industry veterans for every new application
- Automated validation and benchmark comparison alone are insufficient for quality assurance; institutions must build independent verification code paths and anchor outputs against published industry ranges, since flawed assumptions pass internal consistency checks with perfect scores
Disclaimer: The above content is generated by AI and is for reference only.