HypoKG: Evidence-Disciplined Biomedical Hypothesis Generation Beyond Endpoint Knowledge
HypoKG integrates KEGG, Rhea, and UniProt into a unified biochemical knowledge graph to benchmark LLM-based biomedical hypothesis generation Six LLMs generated 13,200 hypotheses across four conditions varying the biological information provided (source enzyme only, full path, endpoints only) Models given only source and disease endpoints produced the most compelling hypotheses but were least evidence-grounded Full biological path information led to "evidence-disciplined reasoning" — hypotheses m
Analysis
TL;DR
- HypoKG integrates KEGG, Rhea, and UniProt into a unified biochemical knowledge graph to benchmark LLM-based biomedical hypothesis generation
- Six LLMs generated 13,200 hypotheses across four conditions varying the biological information provided (source enzyme only, full path, endpoints only)
- Models given only source and disease endpoints produced the most compelling hypotheses but were least evidence-grounded
- Full biological path information led to "evidence-disciplined reasoning" — hypotheses more consistent with known mechanistic relationships
- Shuffling intermediate path steps while keeping endpoints fixed caused a significant drop in evidence grounding (delta = -0.793, p < 0.001), confirming models genuinely used path structure
Why It Matters
This research directly addresses a critical concern in AI for science: whether LLMs are truly reasoning from evidence or merely generating plausible-sounding outputs. For biomedical researchers and AI practitioners, it demonstrates that knowledge graphs serve a dual purpose — identifying novel endpoint pairs and structuring the reasoning process itself. The findings have implications for how we design AI-assisted drug discovery and hypothesis generation pipelines.
Technical Details
- Knowledge Graph Construction: Unified three major biological databases (KEGG, Rhea, UniProt) into a single biochemical knowledge graph
- Benchmark Design: 550 paths connecting enzyme sources to rare disease endpoints, generating 13,200 hypotheses from six LLMs under four experimental conditions
- Evaluation Framework: Expert-derived five-criterion rubric scored on a 1-5 scale per criterion, assessing hypothesis quality and evidence grounding
- Ablation Study: Shuffled intermediate path steps while preserving endpoints to isolate the effect of path structure on reasoning quality
- Key Metric: Evidence grounding score showed statistically significant degradation (delta = -0.793, p < 0.001) when path structure was disrupted
Industry Insight
- Knowledge graphs should be viewed as reasoning scaffolds, not just knowledge repositories — their structural information actively shapes LLM reasoning quality in scientific domains
- There is a trade-off between hypothesis creativity and evidence grounding: minimal information yields more compelling but less reliable hypotheses, while full mechanistic paths produce more disciplined but potentially less novel outputs
- For AI-driven drug discovery pipelines, combining both approaches — using endpoint-only generation for ideation and path-guided generation for validation — could optimize the balance between novelty and reliability
Disclaimer: The above content is generated by AI and is for reference only.