I Tried to Prove DocLang Beats Markdown for PDF→LLM. The Data Said Otherwise.
A converter-independent experiment comparing Markdown vs DocLang for PDF-to-LLM pipelines found identical answer accuracy (100% on facts, 87.5% on structure) across both formats when the full document fits in context DocLang costs 1.5×–2.6× more tokens than Markdown (16,232 vs 6,348 tokens) due to XML wrapper overhead and bounding-box location elements, with zero accuracy gain on this document The DocLang output was structurally flat because the parser failed to recover exhibit hierarchy from th
Analysis
TL;DR
- A converter-independent experiment comparing Markdown vs DocLang for PDF-to-LLM pipelines found identical answer accuracy (100% on facts, 87.5% on structure) across both formats when the full document fits in context
- DocLang costs 1.5×–2.6× more tokens than Markdown (16,232 vs 6,348 tokens) due to XML wrapper overhead and bounding-box location elements, with zero accuracy gain on this document
- The DocLang output was structurally flat because the parser failed to recover exhibit hierarchy from the scanned PDF, meaning DocLang carried no structural advantage over Markdown on this input
- Markdown outperformed DocLang on localization for structure questions (75% vs 62.5%), with bounding-box elements actively distracting the model
- DocLang's value lies in retrieval over large corpora, documents with recovered structure, round-trip fidelity for complex elements (tables, forms), and pixel-accurate provenance citations — not in single-document full-context Q&A
Why It Matters
This experiment directly addresses a critical decision point for anyone building RAG systems or document Q&A pipelines: the choice of text representation between the mature, lightweight Markdown format and the newer AI-native DocLang. The findings challenge the assumption that structured markup automatically yields better LLM performance, and instead show that representation choice must be matched to the specific use case — full-context Q&A favors Markdown, while retrieval, provenance, and complex document elements favor DocLang.
Technical Details
- Experimental design: A single 15-page scanned construction RFP PDF was parsed once into a DoclingDocument, then serialized into three arms (Markdown, native DocLang, DocLang without location elements) using docling-core's first-party serializers — eliminating converter-maturity bias
- Model and evaluation: GPT-5.4-mini via Azure OpenAI in JSON mode with the full document in the system prompt; two question sets (15 fact questions × 3 trials, 8 structure questions × 2 trials) graded by an LLM judge against gold answers and gold locations
- Token overhead breakdown: Native DocLang at 16,232 tokens (2.56× Markdown) driven primarily by
<location>bounding-box elements (four per content block); even stripped of locations, DocLang remained 1.46× larger due to XML tag overhead (~46% over Markdown's##and blank-line conventions) - Localization results: DocLang tied Markdown on fact-set location accuracy (86.7%) but dropped to 73.3% without location elements; on structure questions, DocLang's bbox elements actively hurt performance (62.5% vs Markdown's 75%)
- Root cause of flat output: Docling's layout model parsed the scanned PDF without recovering exhibit hierarchy — all headings were serialized as
level="2"with no<group>containers, and running page headers were repeated ~12 times as genuine heading elements
Industry Insight
- Don't default to DocLang for full-context document Q&A: If your use case fits a single document in the context window, Markdown is the strictly dominant choice — same accuracy, significantly lower cost and latency. Reserve DocLang for cases where its structural features are actually exercised by the parser.
- Parser quality is the bottleneck, not the format: DocLang's theoretical advantages (hierarchical grouping, addressable chunks, provenance) are entirely dependent on the upstream parser recovering that structure. Invest in parser tuning or post-processing (e.g., rebuilding section trees for born-digital PDFs) before expecting format-level gains.
- DocLang's real competitive edge is retrieval and fidelity, not generation: For corpus-scale RAG where structural chunk boundaries prevent running-header bleed, for documents with tables/forms/checkboxes that Markdown mangles, and for applications requiring pixel-accurate citations back to source PDFs, DocLang's token overhead is justified — this experiment explicitly did not test those regimes.
Disclaimer: The above content is generated by AI and is for reference only.