Trajectory-Aware Retrieval Agents for Temporal Decision-Making
The paper introduces TLM (Trajectory Language Model), a closed-loop agentic framework for decision-making from long-form, temporally structured text using LLMs. TLM iteratively refines evidence sets using SHAP-guided feedback and employs a latent growth curve model (LGCM) over retrieved chunk embeddings to detect trajectory trends, turning points, and information gaps. Under a scorer-calibration assumption, the iterative refinement procedure is monotonically non-decreasing in the probability ass
Analysis
TL;DR
- The paper introduces TLM (Trajectory Language Model), a closed-loop agentic framework for decision-making from long-form, temporally structured text using LLMs.
- TLM iteratively refines evidence sets using SHAP-guided feedback and employs a latent growth curve model (LGCM) over retrieved chunk embeddings to detect trajectory trends, turning points, and information gaps.
- Under a scorer-calibration assumption, the iterative refinement procedure is monotonically non-decreasing in the probability assigned to the correct label.
- TLM substantially outperforms zero-shot LLM baselines and standard retrieval-augmented approaches on medical question answering and yields consistent, economically meaningful gains on earnings call surprise prediction and overnight stock gap prediction.
Why It Matters
This work addresses a critical limitation in standard RAG pipelines: the fragmentation of chronological context, which discards temporal structure essential for accurate decision-making. By introducing a trajectory-aware framework with interpretable trend detection and iterative refinement, TLM offers a significant advancement for applications in domains like healthcare and finance where temporal dynamics are paramount.
Technical Details
- Framework: TLM is a closed-loop agentic framework that iteratively refines evidence sets using SHAP-guided feedback to improve decision-making from temporally structured text.
- Latent Growth Curve Model (LGCM): The key technical innovation is the application of LGCM over retrieved chunk embeddings, enabling interpretable detection of trajectory trends, turning points, and information gaps.
- Scorer-Calibration Assumption: The paper assumes scorer-calibration, under which the iterative refinement procedure is monotonically non-decreasing in the probability assigned to the correct label.
- Evaluation Tasks: TLM is evaluated on three temporally grounded decision tasks: medical question answering, earnings call surprise prediction, and overnight stock gap prediction.
- Performance: TLM outperforms zero-shot LLM baselines and standard retrieval-augmented approaches on the medical task and achieves consistent, economically meaningful gains on the financial tasks.
Industry Insight
- Temporal Awareness in Decision-Making: The success of TLM highlights the importance of preserving and leveraging temporal structure in decision-making tasks, particularly in domains like healthcare and finance where time-sensitive information is critical.
- Interpretable AI: The use of LGCM for trajectory detection provides an interpretable mechanism for understanding and improving decision-making processes, which is essential for building trust in AI systems.
- Iterative Refinement: The iterative refinement approach guided by SHAP feedback offers a robust method for enhancing the accuracy and reliability of LLM-based decision-making systems, suggesting potential applications in other domains requiring temporal reasoning.
Disclaimer: The above content is generated by AI and is for reference only.