Toward Automated Detection of Documentation Inconsistencies in Electronic Health Records 面向电子健康记录中文档不一致性的自动检测

TL;DR

• A two-stage LLM pipeline using Gemini 2.5 Pro and Gemini 2.5 Flash was applied to detect documentation inconsistencies in 3,000 MIMIC-IV discharge summaries.
• The pipeline identified 3,460 candidate inconsistencies affecting 69.7% of admissions across domains like demographics, allergies, diagnoses, medications, and lab results.
• Expert review revealed recurring failure modes due to lack of temporal reasoning, evolving-diagnosis context, and outpatient-prescribing knowledge in general-domain LLMs.
• The study proposes a graded ontology for inconsistency classification (from strict contradiction to ambiguity) with a schema based on category, section, domain, and inconsistency axis.
• This work establishes a methodological framework for large-scale EHR inconsistency detection but highlights the need for domain-specific enhancements to improve reliability.

Why It Matters

This research is critical for AI practitioners and healthcare informatics teams aiming to deploy LLMs in clinical settings, as it reveals both the potential and limitations of general-purpose models in detecting subtle documentation errors that could impact patient safety. The findings underscore the necessity of integrating domain-specific reasoning capabilities into healthcare AI systems to reduce false positives/natives and ensure trustworthy automated quality assurance tools. Additionally, the proposed ontology provides a structured approach for evaluating and improving consistency-checking algorithms in electronic health records.

Technical Details

• Pipeline Architecture: Two-stage process: (1) Open-ended candidate identification using Gemini 2.5 Pro to flag potential inconsistencies; (2) Context-grounded verification via Gemini 2.5 Flash to assess flagged pairs against surrounding clinical context.
• Dataset: 3,000 randomly sampled discharge summaries from MIMIC-IV-Note, a widely used public dataset containing de-identified health data from ICU patients at Beth Israel Deaconess Medical Center.
• Inconsistency Domains Covered: Demographics, allergies, procedures, diagnoses, laboratory values, medications, and care planning—each representing high-stakes areas where discrepancies may compromise clinical decision-making or regulatory compliance.
• Failure Modes Identified: Temporal reasoning gaps (e.g., failing to account for changes over time), insufficient understanding of evolving diagnostic criteria during hospitalization, and unawareness of standard outpatient prescribing practices not reflected in inpatient notes.
• Ontology Proposal: A hierarchical classification system ranging from “strict contradiction” (clear factual conflict) through intermediate levels like “partial mismatch” to “ambiguity” (insufficient information to determine truth value), annotated with metadata fields including source section, clinical domain, and type of inconsistency axis (e.g., value mismatch, missing entity).

Industry Insight

Healthcare organizations investing in AI-driven documentation quality control should prioritize hybrid approaches combining general LLMs with specialized modules trained on longitudinal patient trajectories and domain-specific guidelines to mitigate current blind spots. Future development efforts must focus on embedding temporal awareness and contextual adaptability into verification stages to handle dynamic clinical narratives effectively. Furthermore, adopting standardized ontologies for inconsistency categorization—as proposed here—will enable cross-institutional benchmarking, facilitate iterative model improvement, and support regulatory alignment with emerging standards for safe AI deployment in medicine.

TL;DR

• 研究提出利用双阶段大语言模型（LLM）管道自动检测电子健康记录（EHR）出院摘要中的内部文档不一致性。
• 在3,000份MIMIC-IV-Note数据上，该流程识别出3,460个候选不一致项，覆盖近70%的入院病例，涉及人口统计、过敏、诊断、用药等多个临床领域。
• 专家复核发现模型在需时序推理、动态诊断演变或门诊处方知识等场景中易出现重复性失败模式。
• 论文构建了一个分级本体论框架，用于区分严格矛盾与模糊冲突，并按类别、章节、领域和不一致轴对标记案例进行结构化表征。
• 本研究为后续大规模、经过验证的EHR不一致性分析奠定了方法论基础与概念框架。

为什么值得看

该工作揭示了通用域LLM在真实医疗文本中暴露系统性文档缺陷的能力，同时明确其局限性所在——缺乏临床上下文理解与时间逻辑推理能力，这对AI辅助医疗质控系统的开发具有关键指导意义。它不仅展示了NLP技术在提升患者安全方面的潜力，也强调了人机协同验证机制的必要性，是连接前沿模型能力与实际临床需求的重要桥梁。

技术解析

• 采用两阶段LLM流水线：第一阶段使用Gemini 2.5 Pro进行开放式候选不一致识别，第二阶段用更轻量级的Gemini 2.5 Flash执行基于上下文的验证，兼顾效率与准确性。
• 数据来源为MIMIC-IV-Note数据库中的3,000份随机抽样出院摘要，涵盖多种临床文档类型，具备现实世界复杂性与多样性。
• 人工由临床专家对部分输出结果进行审核，以评估模型表现并识别系统性错误模式，确保评估的可靠性与临床相关性。
• 提出一种分级本体论（graded ontology），将不一致从“严格矛盾”到“语义模糊”连续谱系化，并设计标准化schema记录每个案例的分类属性（如所属章节、医学领域、不一致维度）。
• 实验结果表明，尽管模型能广泛捕捉潜在冲突，但在涉及病程演进、跨时段信息比对或非标准处方惯例时仍显不足，凸显当前LLM在深度临床推理上的短板。

行业启示

• 医疗机构应重视自动化文档一致性检查工具的研发与应用，将其作为提升病历质量、减少医疗差错的重要技术手段，尤其适用于高风险科室如急诊、重症监护等。
• AI开发者需在模型训练中引入更多纵向临床知识图谱与时间感知机制，增强对疾病进展、治疗响应及医嘱变更序列的理解能力，以降低误报率并提高实用性。
• 未来系统必须采用“AI初筛+专家复核”的混合架构，既发挥规模化处理能力，又保留人类判断的最终否决权，从而在保证安全性的前提下实现高效落地。