RAG Workflow and Loop Engineering: The Dispatcher That Decides When to Loop and When to Stop
The article introduces a composite RAG pipeline (`pdf_qa_loop`) that composes multiple retrieval/generation patterns into a single orchestrated workflow with bounded iteration loops A deterministic Python dispatcher selects which patterns fire based on parsed questions and document profiles, keeping control logic in code rather than delegating to LLM agents Real enterprise questions often trigger multiple patterns simultaneously (e.g., TOC retrieval + listing aggregation + synthesis), requiring
Analysis
TL;DR
- The article introduces a composite RAG pipeline (
pdf_qa_loop) that composes multiple retrieval/generation patterns into a single orchestrated workflow with bounded iteration loops - A deterministic Python dispatcher selects which patterns fire based on parsed questions and document profiles, keeping control logic in code rather than delegating to LLM agents
- Real enterprise questions often trigger multiple patterns simultaneously (e.g., TOC retrieval + listing aggregation + synthesis), requiring a unified stop-condition mechanism instead of ad-hoc special cases
- The system builds on four foundational bricks: document parsing, question parsing, retrieval, and generation, with typed feedback fields (
AnswerWithEvidence) driving loop continuation decisions - This approach sits two rungs below agentic LLM-controlled loops on a five-level capability ladder, prioritizing reproducibility, auditability, and explicit code over model-orchestrated control
Why It Matters
Enterprise RAG systems face a critical scalability problem: as question complexity grows, bolted-on special-case patterns become unmaintainable. This article provides a principled architecture for composing iterative retrieval patterns with bounded loops and code-governed dispatch, offering a production-ready alternative to fully agentic approaches that sacrifice auditability for flexibility. For AI practitioners building document intelligence pipelines, it demonstrates how to separate pattern selection and iteration control from the LLM itself, enabling reproducible, reviewable workflows suitable for regulated environments.
Technical Details
- Dispatcher Architecture: A single orchestrator function (
pdf_qa_loop) takes aParsedQuestionandDocumentProfile, deterministically selects which patterns to activate (e.g., TOC retrieval, listing aggregation, two-hop reference resolution, synthesis), and composes them into a bounded workflow - Bounded Feedback Loops: Each pattern carries its own iteration mechanic (re-retrieval, re-generation, LLM flags), but loop continuation is governed by typed feedback fields on
AnswerWithEvidenceobjects—specifically flags likeis_context_completeanddid_parse_hold—with ashould_continuefunction that cuts loops when candidates stop moving - Pattern Toolkit Composition: Four patterns are discussed: TOC retrieval (landing on correct document sections), listing aggregation (enumerating all items, not just top-cited), synthesis (selecting from enumerated results), and two-hop reference resolution (following cross-references); patterns are sub-functions within their respective brick modules, never leaking internals
- Layered Architecture: The system maintains strict separation between the composition layer (dispatcher, feedback machinery, flow per intent) and the four brick modules (parsing, retrieval, generation); communication occurs only through typed objects, not shared internals
- Five-Rung Capability Ladder: Baseline (single-pass keyword retrieval) → Upgraded (relational parse + TOC routing with feedback fields) → Workflow (this article: bounded loops + dispatcher) → Multi-intent (chat entry classification routing to different pipelines) → Agentic (LLM controls the loop); the article explicitly stops at rung three to preserve code-governed reproducibility
Industry Insight
- Agent hype vs. production reality: The article deliberately positions itself two rungs below agentic LLM control, signaling that for regulated enterprise contexts (compliance, legal, government), code-governed pipelines with explicit iteration bounds are more deployable than model-orchestrated loops—practitioners should evaluate where their use case falls on this ladder before investing in agentic architectures
- Pattern composition is the unsolved scaling problem: As questions stack patterns (e.g., listing obligations plus two-hop reference resolution), the dispatcher and stop-condition logic become the critical differentiator; teams should invest in a reusable pattern toolkit with well-defined typed interfaces rather than hardcoding question-specific branches
- Feedback fields as the key abstraction: The
AnswerWithEvidencetyped object with flags likeis_context_completedecouples pattern logic from loop control, enabling new patterns to be added without rewriting iteration machinery—a design pattern worth adopting for any iterative RAG system
Disclaimer: The above content is generated by AI and is for reference only.