EO-Agents: A Three-Agent LLM Pipeline for Earth Observation Hypothesis Generation
EO-Agents introduces a novel pipeline for Earth observation hypothesis generation by grounding LLMs in the NASA Earth Observation Knowledge Graph rather than unstructured text. The system utilizes a heterogeneous graph neural network to rank candidate dataset pairings based on historical co-usage, followed by a three-agent LLM process for filtering, generating, and evaluating hypotheses. Applied to 1,475 NASA datasets, the pipeline produced 160 scientifically coherent hypotheses across diverse d
Analysis
TL;DR
- EO-Agents introduces a novel pipeline for Earth observation hypothesis generation by grounding LLMs in the NASA Earth Observation Knowledge Graph rather than unstructured text.
- The system utilizes a heterogeneous graph neural network to rank candidate dataset pairings based on historical co-usage, followed by a three-agent LLM process for filtering, generating, and evaluating hypotheses.
- Applied to 1,475 NASA datasets, the pipeline produced 160 scientifically coherent hypotheses across diverse domains like ecohydrology and glaciology, with novel pairings rated nearly as plausible as real-world literature co-usages.
- A factorial experiment comparing GPT-5.2 and Claude Sonnet 4.6 revealed that while hypothesis rankings are stable, absolute scores vary significantly by judge identity, exposing limitations in single-judge LLM evaluation.
Why It Matters
This research demonstrates a significant shift from text-only LLM applications to structured, knowledge-graph-grounded scientific discovery, offering a scalable method for identifying novel research opportunities in complex domains. For AI practitioners, it highlights the critical importance of hybrid architectures combining graph neural networks with generative models to ensure factual grounding and structural coherence in hypothesis generation. Furthermore, the findings on judge variability provide essential guidance for designing robust evaluation frameworks in automated scientific reasoning tasks.
Technical Details
- Architecture: The pipeline integrates a heterogeneous graph neural network (GNN) trained on historical co-usage relations within the NASA Earth Observation Knowledge Graph to rank dataset pairings, followed by a multi-agent LLM system.
- Agent Roles: The three-agent LLM pipeline performs distinct functions: filtering irrelevant candidates, generating structured research hypotheses, and evaluating their scientific plausibility.
- Scope and Results: Tested on 1,475 NASA datasets, the system generated 160 hypotheses spanning five major Earth-science domains, including aerosol-cloud interactions and stratospheric chemistry.
- Evaluation Methodology: A 2x2x2 factorial experiment was conducted using GPT-5.2 and Claude Sonnet 4.6 to assess stability and bias in LLM-based evaluation, revealing that relative rankings are consistent but absolute scores are model-dependent.
Industry Insight
- Structured Grounding is Key: For scientific AI applications, relying solely on unstructured text limits novelty and accuracy; integrating knowledge graphs provides necessary constraints and contextual depth for high-quality hypothesis generation.
- Hybrid Models Outperform Pure LLMs: Combining symbolic methods (like GNNs) with neural generative models offers a robust path for handling complex, relational data in specialized industries such as climate science and healthcare.
- Evaluation Frameworks Need Multi-Judge Consensus: The variance in absolute scores between different LLM judges suggests that automated evaluation pipelines must employ multiple models or consensus mechanisms to avoid bias and ensure reliable metric reporting.
Disclaimer: The above content is generated by AI and is for reference only.