BatchDAG: LLM-Planned Execution Graphs for Scalable Ad-Hoc Analysis Over Enterprise Data
BatchDAG introduces an LLM-planned Directed Acyclic Graph (DAG) architecture to replace sequential tool calls with parallel, deterministic execution for enterprise-scale ad-hoc analysis. The system utilizes entity-aware batching to group data by logical entities before fan-out, reducing LLM API calls by up to 47x while maintaining high throughput. Structured JSON intermediates between graph nodes reduce hallucinations by 27% compared to prose-based summaries, ensuring better data provenance and
Analysis
TL;DR
- BatchDAG introduces an LLM-planned Directed Acyclic Graph (DAG) architecture to replace sequential tool calls with parallel, deterministic execution for enterprise-scale ad-hoc analysis.
- The system utilizes entity-aware batching to group data by logical entities before fan-out, reducing LLM API calls by up to 47x while maintaining high throughput.
- Structured JSON intermediates between graph nodes reduce hallucinations by 27% compared to prose-based summaries, ensuring better data provenance and accuracy.
- In production tests involving over 50,000 meeting transcripts, BatchDAG processes complex queries in under 60 seconds with low per-query costs ($0.02-$0.24).
- The approach serves as a general-purpose orchestration layer that matches expert-designed pipeline quality without requiring manual workflow engineering for each use case.
Why It Matters
This research addresses critical scalability bottlenecks in current LLM applications, specifically context overflow and linear latency inherent in sequential agent frameworks like ReAct. By shifting from narrative-based reasoning to structured, graph-based execution, BatchDAG offers a viable path for deploying LLMs in high-volume, data-intensive enterprise environments where speed, cost, and traceability are paramount.
Technical Details
- Architecture: An LLM planner generates a typed DAG comprising SQL queries, semantic searches, in-memory transforms, and parallel fan-outs, which are then executed by a deterministic engine using topological-wave parallelism.
- Optimization: Entity-aware batching groups rows by logical entity prior to processing, significantly minimizing redundant LLM invocations and reducing call volume by up to 47x.
- Data Flow: The system enforces structured JSON data flow between nodes, which was shown in ablation studies to cut hallucination rates by 27% compared to unstructured prose outputs.
- Performance Metrics: The planner achieved a 98.8% valid-DAG generation rate across 300 trials, and the system demonstrated superior provenance with a 77% transcript evidence rate against baseline methods.
Industry Insight
- Enterprises should prioritize orchestration layers that decouple planning from execution, allowing for parallel processing of large datasets rather than relying on slow, sequential agent loops.
- Implementing strict schema enforcement (e.g., JSON intermediates) between LLM steps is a highly effective strategy for mitigating hallucinations and improving auditability in production systems.
- The cost-efficiency gains from batching and parallelization suggest that LLM-driven analytics can become economically viable for massive datasets, provided the execution graph is optimized for entity-level grouping.
Disclaimer: The above content is generated by AI and is for reference only.