Prompt, Context, Loop: The Three Engineering Layers Every RAG System Is Built On
The article introduces a three-layer framework for enterprise RAG systems: prompt engineering (the LLM call itself), context engineering (filling the model's context window), and loop engineering (control flow around multiple calls) The dominant narrative that these layers evolved sequentially is misleading; all three patterns existed simultaneously since the early LLM era, with names installing years after the practices were already in production The real driver of change is bottleneck migratio
Analysis
TL;DR
- The article introduces a three-layer framework for enterprise RAG systems: prompt engineering (the LLM call itself), context engineering (filling the model's context window), and loop engineering (control flow around multiple calls)
- The dominant narrative that these layers evolved sequentially is misleading; all three patterns existed simultaneously since the early LLM era, with names installing years after the practices were already in production
- The real driver of change is bottleneck migration: as models improve at one layer, the production constraint shifts upward to the next layer
- Three forces push the bottleneck up the stack: model improvements reduce prompt engineering needs, longer context windows will reduce context engineering demands, and agent reliability over many turns becomes the new frontier
- The article serves as a manifesto for an "Enterprise Document Intelligence" series with companion code at doc-intel/notebooks-vol1 on GitHub
Why It Matters
This framework gives AI practitioners a clearer mental model for diagnosing production RAG failures and understanding why certain engineering disciplines gain prominence at different times. The bottleneck-moves insight helps teams avoid premature optimization by recognizing which layer is currently the rate-limiting factor in their systems.
Technical Details
- Prompt engineering involves the system message, user instructions, and output schema that define the LLM call itself; it was the dominant discipline during 2022-2023 with GPT-3.5 and ChatGPT
- Context engineering manages what fills the model's finite context window through retrieval, compression, and isolation strategies; LangChain's four canonical strategies (write, select, compress, isolate) formalized practices that existed since the original RAG paper (Lewis et al., 2020)
- Loop engineering controls what happens around the call: trigger conditions, stop criteria, failure recovery, and multi-agent verification; the term installed in May 2026 with Boris Cherny's work and Anthropic's Dynamic Workflows launch
- Historical patterns predate their names by years: ReAct (October 2022) predates loop engineering by 3+ years, AutoGPT (March 2023), Reflexion (NeurIPS 2023), and Ralph Loop (July 2025) all represent loop engineering patterns before the term existed
- Companion notebooks at doc-intel/notebooks-vol1 demonstrate each layer end-to-end on real PDFs
Industry Insight
- Teams should diagnose their current bottleneck rather than chase the latest named discipline; if prompts are working well, the constraint has likely shifted to context management or loop reliability
- Model providers will continue to erode the value of lower-layer engineering as capabilities improve, making loop engineering and multi-turn agent reliability the increasingly important differentiator
- The "evolution" narrative is marketing framing; practitioners should recognize that all three layers are always present and focus investment on whichever layer is currently the production bottleneck
Disclaimer: The above content is generated by AI and is for reference only.