Automata from Agent Traces: Failure and Next-Step Prediction
LLM-based agent traces are collapsed into a single compact finite-state machine (FSM) that captures cross-run behavioral topology, addressing the opacity of multi-step agent execution FSMs built across twelve public datasets are remarkably compact (7-43 states), replay held-out data with >=0.997 fitness, and construct in milliseconds For next-step prediction, FSM-state context outperforms Agent Workflow Memory on every ground-truth-matched dataset For failure prediction, per-state behavioral fea
Analysis
TL;DR
- LLM-based agent traces are collapsed into a single compact finite-state machine (FSM) that captures cross-run behavioral topology, addressing the opacity of multi-step agent execution
- FSMs built across twelve public datasets are remarkably compact (7-43 states), replay held-out data with >=0.997 fitness, and construct in milliseconds
- For next-step prediction, FSM-state context outperforms Agent Workflow Memory on every ground-truth-matched dataset
- For failure prediction, per-state behavioral features achieve held-out AUROC up to 0.94, with an online monitor capable of ranking failing runs above passing ones from partial traces to trigger early stopping
- The behavioral topology appears shaped more by the deployment harness than by the underlying LLM, offering a model-agnostic structural primitive for safety auditing and runtime monitoring
Why It Matters
This work provides a practical, model-agnostic approach to making LLM agent behavior interpretable and monitorable at scale, which is a critical bottleneck for deploying agents in safety-sensitive environments. By distilling unstructured traces into compact FSMs, practitioners gain a structural substrate that enables both predictive analytics and real-time safety interventions without requiring per-model customization.
Technical Details
- The core method collapses an entire corpus of agent execution traces into a single finite-state machine, recovering shared cross-run topology that per-trace or success-only approaches miss
- Evaluated across twelve public datasets, producing FSMs with only 7-43 states that achieve >=0.997 fitness on held-out replay with near-identical topology across data splits
- Next-step prediction leverages FSM-state context as a feature representation, outperforming Agent Workflow Memory across all ground-truth-matched datasets
- Failure prediction uses per-state behavioral features to reach AUROC up to 0.94 on held-out data; an online monitor can rank failing runs above passing ones from partial traces, enabling early stopping well before task completion
- The approach is model-agnostic, suggesting the behavioral topology is determined primarily by the deployment harness rather than the specific LLM powering the agent
Industry Insight
- Safety auditing and runtime monitoring for LLM agents can leverage compact FSM abstractions as a lightweight, model-agnostic primitive, reducing the need for expensive per-model analysis pipelines
- Early stopping triggered by partial-trace failure prediction could significantly reduce compute waste in agent deployments, especially in high-volume or cost-sensitive production environments
- The finding that deployment harnesses shape behavioral topology more than the LLM itself suggests that agent safety improvements may be more effectively targeted at orchestration and tooling layers rather than model selection alone
Disclaimer: The above content is generated by AI and is for reference only.