From Causal Plausibility to Causal Reliability: Evaluating LLMs as Calibrated Direct Causal-Edge Classifiers
LLM-based causal judgments are strongly recall-dominant, producing overly dense graphs with many false-positive edges; prompting shifts the precision-recall trade-off but does not resolve overprediction LLMs capture causal relatedness but fail to reliably identify directness or orientation, misclassifying 40.0% of indirect and 36.0% of reversed non-edges as direct edges 80.8% and 84.6% of false positives from indirect and reversed non-edges receive verbalized confidence of at least 80%, revealin
Analysis
TL;DR
- LLM-based causal judgments are strongly recall-dominant, producing overly dense graphs with many false-positive edges; prompting shifts the precision-recall trade-off but does not resolve overprediction
- LLMs capture causal relatedness but fail to reliably identify directness or orientation, misclassifying 40.0% of indirect and 36.0% of reversed non-edges as direct edges
- 80.8% and 84.6% of false positives from indirect and reversed non-edges receive verbalized confidence of at least 80%, revealing substantial overconfidence in structurally incorrect predictions
- Conventional confidence estimates (logit-based) are unreliable, collapsing near 1.0 regardless of correctness, while cross-prompt and cross-model agreement show better calibration though not statistically significant after Holm correction
- Benchmark-familiarity audit flags potential data contamination in five model-dataset pairs involving AsiaM, reinforcing the conclusion that LLMs should serve as externally validated soft causal priors rather than direct evidence of causal structure
Why It Matters
This work directly addresses a growing practice in the AI community where LLMs are used to supply prior causal knowledge for structural causal discovery, yet their reliability as direct edge classifiers remains unproven. For practitioners building causal models or integrating LLMs into scientific workflows, the findings serve as a critical caution against trusting raw LLM causal judgments and their confidence scores without external validation. The results reshape how researchers should deploy LLMs in causal inference pipelines, shifting expectations from direct structural evidence to calibrated soft priors.
Technical Details
- Evaluation scope: 12 instruction-tuned open-weight LLMs tested across 6 benchmark causal graphs, 5 prompting strategies, and 4 confidence estimation methods (verbalized, logit-based, cross-prompt agreement, cross-model agreement)
- Protocol: Language-only pairwise causal-edge classification under a structured evaluation framework comparing LLM predictions against published reference graphs
- Key metrics: Precision-recall analysis, calibration assessment, and error classification distinguishing indirect edges, reversed edges, and other non-edges
- Confidence analysis: Logit-based confidence was found to collapse near 1.0 irrespective of prediction correctness; verbalized confidence showed 80.8% and 84.6% of false positives receiving scores above 80% for indirect and reversed non-edges respectively
- Data contamination audit: Identified potential benchmark familiarity in 5 model-dataset pairs, all involving the AsiaM dataset, raising concerns about evaluation validity
Industry Insight
- Researchers and engineers should treat LLM-generated causal edges as hypothesis-generating priors requiring independent empirical validation rather than as standalone evidence; this applies to any workflow using LLMs for scientific causal discovery
- Confidence calibration remains a critical unresolved challenge—logit-based and verbalized confidence scores from current LLMs are unreliable indicators of prediction correctness, suggesting the need for dedicated calibration methods before deploying LLMs in high-stakes causal reasoning tasks
- The diminishing returns of model scale on larger graphs and the lack of statistical significance in agreement-based calibration improvements indicate that simply scaling up models will not solve the fundamental overconfidence and overprediction problems in LLM-based causal edge classification
Disclaimer: The above content is generated by AI and is for reference only.