Faithful by Design: Evaluating and Improving LLM-Generated Clinical Trial Summaries for Multi-Stakeholder Audiences
Introduces a benchmark evaluation framework to measure the faithfulness of LLM-generated clinical trial summaries across three distinct stakeholder audiences. Identifies "Unsupported Claims" as the dominant failure mode for GPT-4o, Claude Sonnet 4.6, and Gemini 2.5 Flash, with a mean annotation score of 1.55/3. Demonstrates that a knowledge-graph-augmented retrieval system yields statistically significant improvements in NLI-based faithfulness scores (p < 0.0001). Reveals model-specific improvem
Analysis
TL;DR
- Introduces a benchmark evaluation framework to measure the faithfulness of LLM-generated clinical trial summaries across three distinct stakeholder audiences.
- Identifies "Unsupported Claims" as the dominant failure mode for GPT-4o, Claude Sonnet 4.6, and Gemini 2.5 Flash, with a mean annotation score of 1.55/3.
- Demonstrates that a knowledge-graph-augmented retrieval system yields statistically significant improvements in NLI-based faithfulness scores (p < 0.0001).
- Reveals model-specific improvement pathways: GPT-4o reduces contradictions, while Claude Sonnet 4.6 and Gemini 2.5 Flash increase entailment.
Why It Matters
This research addresses critical safety concerns in healthcare AI by providing a rigorous method to evaluate and mitigate hallucinations in high-stakes clinical contexts. It offers actionable insights for developers aiming to integrate LLMs into medical workflows, emphasizing the need for audience-specific evaluation and robust retrieval mechanisms. The findings highlight that generic summarization is insufficient for clinical accuracy, necessitating specialized architectural interventions like knowledge graphs.
Technical Details
- Evaluation Framework: Utilizes 200 stratified clinical trials from the Aggregate Analysis database, assessed via audience-specific prompt templates and a six-dimension faithfulness annotation schema.
- Baseline Models: Tests GPT-4o, Claude Sonnet 4.6, and Gemini 2.5 Flash, generating 1,800 summaries scored using a cross-encoder natural language inference (NLI) model.
- Failure Mode Analysis: "Unsupported Claims" was the primary error source across all models, indicating a systemic issue with factual grounding in current generative architectures.
- Intervention: Implementation of a knowledge-graph-augmented retrieval system that significantly enhanced faithfulness metrics, proving the efficacy of structured data integration in reducing hallucinations.
Industry Insight
- Prioritize Retrieval-Augmented Generation (RAG): For clinical applications, standard LLM prompting is inadequate; integrating structured knowledge bases is essential for maintaining factual integrity.
- Audience-Specific Optimization: Summarization strategies must be tailored to the target audience (providers vs. patients), as faithfulness requirements and failure modes may vary by stakeholder.
- Model Selection Strategy: Different models exhibit distinct improvement patterns when augmented; GPT-4o benefits from contradiction reduction, while others benefit from entailment increases, suggesting a need for model-specific tuning in production pipelines.
Disclaimer: The above content is generated by AI and is for reference only.