Show HN: Record, replay, and improve AI agents in production
Kitaru is a self-hosted, framework-agnostic runtime for autonomous agents that sits between the model and the harness, providing execution recording and replay capabilities. It captures every step of an agent run—including model calls, tool usage, and decisions—as versioned, replayable checkpoints stored in user-owned object storage. Key features include crash recovery via cached checkpoints, pause/resume functionality for human-in-the-loop workflows, and isolated execution for heavy or risky st
Analysis
TL;DR
- Kitaru is a self-hosted, framework-agnostic runtime for autonomous agents that sits between the model and the harness, providing execution recording and replay capabilities.
- It captures every step of an agent run—including model calls, tool usage, and decisions—as versioned, replayable checkpoints stored in user-owned object storage.
- Key features include crash recovery via cached checkpoints, pause/resume functionality for human-in-the-loop workflows, and isolated execution for heavy or risky steps.
- The system allows developers to swap models or inputs during replay to test changes before shipping, enabling evidence-based optimization rather than guesswork.
- Integration is designed to be non-invasive, allowing existing agents built on frameworks like Pydantic AI, LangGraph, or raw Python to be wrapped with minimal code changes.
Why It Matters
This addresses a critical gap in the current AI agent ecosystem: the lack of reliable debugging and reproducibility tools for production-grade autonomous systems. By decoupling the execution layer from the agent logic, Kitaru enables organizations to maintain control over their infrastructure and data while gaining enterprise-grade observability and resilience. This approach reduces the risk associated with deploying complex, multi-step agents by providing a safety net for testing and recovery.
Technical Details
- Architecture: Kitaru operates as a middleware layer, utilizing
@flowand@checkpointdecorators to instrument Python code. It stores artifacts in standard object stores (S3, GCS, Azure Blob) rather than proprietary databases. - Replay and Debugging: The system records typed, versioned artifacts for each step. Users can re-run executions from any checkpoint, overriding specific parameters, models, or tool outputs to simulate different scenarios or debug failures.
- Resource Management: Supports isolated execution where specific checkpoints can run in separate pods or jobs on Kubernetes or cloud providers, keeping orchestration lightweight. It also includes a
wait()function to suspend flows until external inputs (human approval, webhooks) are received. - Integration: Provides adapters for popular frameworks like Pydantic AI (
KitaruAgent) and supports wrapping raw Python or other SDKs (OpenAI, Anthropic) without requiring a full rewrite of the agent logic. - Deployment: Self-hosted via a single server, offering a built-in UI for observability, run inspection, and human-in-the-loop approvals, with no mandatory SaaS control plane.
Industry Insight
- Shift to Infrastructure-Centric Agent Development: As agent complexity grows, the focus is moving from just building logic to managing execution reliability. Organizations should prioritize tools that offer observability and replay capabilities to reduce operational overhead and debugging time.
- Vendor Neutrality as a Strategic Advantage: Framework-agnostic runtimes allow companies to avoid lock-in, enabling them to switch LLM providers or agent frameworks without rebuilding their entire execution pipeline. This flexibility is crucial for cost optimization and risk mitigation.
- Importance of Deterministic Testing in Non-Deterministic Systems: The ability to replay runs with overrides provides a deterministic testing environment for inherently stochastic LLM outputs. Teams should adopt these practices to validate agent improvements rigorously before deployment, ensuring consistent performance and easier root-cause analysis.
Disclaimer: The above content is generated by AI and is for reference only.