A RAG That Says 'Not in This Document' Has to Show Four Kinds of Evidence
A RAG system's most valuable response is sometimes "that is not in this document," but models naturally tend to answer anyway, so honest "not found" responses must be deliberately engineered rather than assumed A confident wrong answer is a bug, while an unverified bare "no answer" is almost as bad—absence claims require defensible evidence, not just a refusal to respond The "four bricks" framework (parsing, retrieval, structured output, and verification) each contribute one piece of evidence to
Analysis
TL;DR
- A RAG system's most valuable response is sometimes "that is not in this document," but models naturally tend to answer anyway, so honest "not found" responses must be deliberately engineered rather than assumed
- A confident wrong answer is a bug, while an unverified bare "no answer" is almost as bad—absence claims require defensible evidence, not just a refusal to respond
- The "four bricks" framework (parsing, retrieval, structured output, and verification) each contribute one piece of evidence to make a "no" verdict trustworthy and auditable
- Parsing produces relational DataFrames (line_df, image_df, page_df, toc_df, cross_ref_df) that serve as the audit trail for absence claims, covering text lines, images, tables, and cross-references
- False negatives in parsing are silently dangerous: the system appears to correctly say "no," but the answer was actually present and simply never detected by the pipeline
Why It Matters
This article addresses one of the most persistent failure modes in enterprise RAG systems: the inability to reliably and defensibly report when information is absent from a document. For AI practitioners building production retrieval systems, this is critical because users will lose trust in a chatbot that either hallucinates answers or gives unverified "I don't know" responses that may simply reflect shallow retrieval. The framework provides a concrete, structured approach to making absence claims auditable and trustworthy.
Technical Details
- Relational parsing output: The
parse_pdffunction produces a structured set of DataFrames rather than unstructured text—line_df(text lines with page numbers, bounding boxes, and line types),image_df(registered images),page_df,toc_df(table of contents entries),cross_ref_df(in-body cross-references), and anobject_registry—each serving as evidence surfaces for absence verification - Coverage aggregation for absence claims: A
parse_coveragedictionary aggregates key metrics (total pages, pages with text, image count, TOC entries, cross-references, registered objects) to define the scope an absence claim must cover, ensuring the system can demonstrate it searched all relevant surfaces - OCR pass on image_df: Charts and figures often contain axis labels and legends as image-embedded text rather than typographic spans; an OCR pass adds an
ocr_textcolumn toimage_dfwithout modifyingline_df, and the coverage report tracks how many images were processed - Table-aware retrieval: Numbers in table cells parse as lines whose neighbors are other cells in the same row, not words above on the page—meaning a keyword sweep can miss relevant data; the
object_registryflags which pages host tables so retrieval can perform column-aware sweeps - Asymmetric cost of parsing errors: False positives (extracting noise) are recoverable downstream, but false negatives (failing to extract a real token) are silent and catastrophic for absence claims—the system appears correct while the answer was actually present but undetected
Industry Insight
- Enterprise RAG systems must treat "I don't know" as a first-class output that requires the same rigor of evidence and verification as affirmative answers—simply refusing to answer is insufficient without an audit trail proving the system searched comprehensively
- The relational parsing approach (treating documents as structured DataFrames rather than text blobs) should be adopted as a standard practice in enterprise document intelligence pipelines, as it enables both affirmative and negative answers to be verified and audited
- Organizations building corporate chatbots should invest in coverage reporting and absence-claim verification as part of their evaluation framework, since user trust erodes not from honest "no answer" responses but from unverifiable ones that may mask shallow or incomplete retrieval
Disclaimer: The above content is generated by AI and is for reference only.