Building Document Structure with Loop Engineering: Recovering a PDF's Outline from Body Typography for RAG
Introduces a fourth TOC reconstruction case that detects headings from body typography when documents lack native outlines or printed tables of contents Implements a cascading detection pipeline with four cases ordered by cost: native outline → contents page with links → printed contents text → body typography reconstruction Uses typographic signals (font size, bold ratio, alignment, numeric prefixes, length, blank-line context) to identify heading candidates from PDF body text Integrates with e
Analysis
TL;DR
- Introduces a fourth TOC reconstruction case that detects headings from body typography when documents lack native outlines or printed tables of contents
- Implements a cascading detection pipeline with four cases ordered by cost: native outline → contents page with links → printed contents text → body typography reconstruction
- Uses typographic signals (font size, bold ratio, alignment, numeric prefixes, length, blank-line context) to identify heading candidates from PDF body text
- Integrates with existing Article 5B relational data model, producing standard toc_df output compatible with retrieval and chunking pipelines
- Validated on the Attention paper (1706.03762v7.pdf), recovering 21 real headings with 3 false positives filtered by LLM validation
Why It Matters
This work addresses a critical gap in enterprise RAG systems: many documents, especially LaTeX-exported papers, lack structured metadata for proper chunking and retrieval. By treating body-typography reconstruction as a detection case rather than a summarization fallback, the pipeline maintains deterministic output shapes and source traceability essential for production document intelligence systems.
Technical Details
- Four-case cascade: The system attempts TOC reconstruction in order of computational cost—native outline extraction, contents page with links, printed contents text analysis, and finally body typography detection—dropping through when earlier cases return insufficient results
- Typographic signal extraction: Uses PyMuPDF span-level typography data (font size, bold ratio, italic, dominant font name) aggregated to line level via
enrich_line_df_with_style(), with six signal functions scoring heading likelihood based on visual prominence and structural cues - Parser flexibility: Handles varying parser capabilities—from rich typography exposure (PyMuPDF) to text-only outputs (EasyOCR, Azure OCR Layout)—with missing typographic columns gracefully degrading to position and text-based signals
- LLM validation loop: After deterministic heading candidate generation, an LLM pass filters false positives, demonstrated achieving 21 true headings plus 3 false positives on the Attention paper before validation filtering
- Standardized output: Produces toc_df in the Article 5B relational data model shape, ensuring downstream retrieval, chunking, and summarization components work unchanged across all four detection cases
Industry Insight
- Enterprise RAG systems should implement graceful degradation for document structure detection, as real-world document quality varies dramatically; the four-case cascade pattern can be adapted to other document intelligence tasks beyond TOC reconstruction
- The distinction between detection and summarization approaches for structure recovery matters for production systems: detection provides fixed output schemas and source traceability that summarization cannot guarantee, critical for auditability in enterprise contexts
- Investing in typographic signal extraction at the parsing layer (span-level font metadata) pays dividends downstream, enabling structure recovery for documents that lack any machine-readable outline or metadata
Disclaimer: The above content is generated by AI and is for reference only.