Multi-Document RAG: A Folder of Unrelated PDFs Is One Long Document with a Nested Outline
When documents in a folder share no common fields, the RAG architecture shifts from indexed tables to a two-level routing system treating the folder as one long document with files as chapters Preparation collapses to two lightweight artefacts: one summary line per file (Level 0) and each file's table of contents (Level 1), eliminating the need for schema extraction Two diagnostic questions determine corpus type: whether documents reference each other, and whether a business user can name a shar
Analysis
TL;DR
- When documents in a folder share no common fields, the RAG architecture shifts from indexed tables to a two-level routing system treating the folder as one long document with files as chapters
- Preparation collapses to two lightweight artefacts: one summary line per file (Level 0) and each file's table of contents (Level 1), eliminating the need for schema extraction
- Two diagnostic questions determine corpus type: whether documents reference each other, and whether a business user can name a shared field with consistent meaning across all documents
- The two-level routing system filters irrelevant files before any page content is read, demonstrated on a 63-file, 4,211-page corpus of NIST, arXiv, and World Bank documents
- This approach trades away filtering, aggregation, and shared vocabulary capabilities that homogeneous corpora enjoy, but requires minimal new machinery
Why It Matters
This article addresses a common real-world RAG challenge: enterprise document repositories rarely consist of uniform records with shared metadata fields. By providing a practical architecture for heterogeneous folders, it gives practitioners a concrete alternative to over-engineering or falling back to naive vector search. The diagnostic framework for classifying corpus types prevents costly architectural mismatches before implementation begins.
Technical Details
- Two-question diagnostic: Documents are classified by (1) whether they cross-reference each other and (2) whether a business user can name a consistently-meaningful shared field; if both answers are no, the folder is treated as a single long document
- Two-level index structure: Level 0 consists of one-line summaries per file written for a router (not a reader), and Level 1 is the parser-returned table of contents for each file, forming a nested outline
- Two-level routing architecture: A routing call operates on the flat file list first, pruning irrelevant documents before any page content is ingested into the context window; the nested outline guides subsequent navigation without ever entering a prompt whole
- Demonstration corpus: 63 public PDFs totaling 4,211 pages — 25 NIST publications (public domain), 31 arXiv papers, and 7 World Bank Commodity Markets Outlook issues (CC BY 3.0 IGO)
- Four failure modes identified: Including the scaling limit where the flat file list approach breaks down, with companion notebook code available at doc-intel/notebooks-vol1
Industry Insight
- Mixed folders are the norm, not the exception; splitting heterogeneous drives into homogeneous and heterogeneous partitions and applying the correct architecture to each avoids the cost of building the wrong system
- Writing summary lines for a router rather than a reader is a subtle but critical design choice that affects retrieval quality — summaries must be discriminative enough to route queries, not informative enough to answer them
- The "folder as one long document" mental model simplifies enterprise RAG design significantly, but practitioners should be aware of the scaling ceiling where flat file lists become a bottleneck and more sophisticated indexing becomes necessary
Disclaimer: The above content is generated by AI and is for reference only.