Decoding AI's Open-Source Course Maps Three Ways to Run an Agent Loop and the Provider Economics Behind Each
Harness engineering matters more than model selection: swapping only the agent harness moved a coding agent from ~30th to top 5 on Terminal-Bench while keeping the same model throughout Paul Iusztin's Decode agent separates a headless core loop (~20 lines via Pydantic AI) from three distinct run modes: interactive online, remote offline, and async online Interactive mode is latency-bound and requires low-latency hosted APIs with a steering queue that drains at MODEL_REQUEST and WOULD_STOP bounda
Analysis
TL;DR
- Harness engineering matters more than model selection: swapping only the agent harness moved a coding agent from ~30th to top 5 on Terminal-Bench while keeping the same model throughout
- Paul Iusztin's Decode agent separates a headless core loop (~20 lines via Pydantic AI) from three distinct run modes: interactive online, remote offline, and async online
- Interactive mode is latency-bound and requires low-latency hosted APIs with a steering queue that drains at MODEL_REQUEST and WOULD_STOP boundaries to prevent input corruption
- Remote and async modes are throughput-bound, making GPU-hour billing significantly cheaper than per-token API pricing (e.g., 1,000 documents cost ~$97 on frontier APIs vs ~$13 on batched serverless GPU)
- Serverless GPU capacity wins over reserved instances when peak-to-average demand ratios (typically 5–10×) exceed reservation discounts (typically 2–5×), with industry utilization often below 30%
Why It Matters
This article reframes a critical design decision for AI practitioners: the agent harness architecture—not the underlying model—can be the dominant factor in agent performance, as demonstrated by the Terminal-Bench results. For teams building production coding agents, understanding which of the three run modes fits a use case directly impacts both latency characteristics and cost structure, with potential savings of 7× or more by matching the billing model to the workload.
Technical Details
- Headless core architecture: The Decode agent uses a minimal ~20-line Pydantic AI definition for the core loop (LLM picks action → tool executes → observation feeds back), while everything else—memory, skills, sandbox, permissions, LSP feedback, compaction—constitutes the harness. For comparison, Claude Code's leaked source shows a ~150-line core loop.
- Interactive mode (Mode 1): Terminal UI wired to a live session in-process with async token streaming. Solves the steering problem via a steering queue with priority gates at MODEL_REQUEST (before next model call) and WOULD_STOP (when turn ends). Three input modes: Plain Enter steers within turn, Alt+Enter queues follow-up, Esc triggers cooperative abort clearing both queues.
- Remote mode (Mode 2): Headless harness runs on Kitaru (ZenML's agent runtime) deployed to GCP with agents executing on Modal. Features step-by-step progress recording for sandbox resume capability and pause-on-human-input without compute consumption. Tools execute in Modal Sandboxes remotely, Docker locally. Metric is throughput per dollar.
- Async mode (Mode 3): Live session hands work to a job queue and returns immediately. Background workflows fan out LLM calls and post results later. Run outlives the client that started it. Pattern suits Slack-triggered agents and background PR review.
- Cost analysis: Frontier API rates ($3M input, $15M output) yield ~$97 for 1,000 documents at 30K input tokens each vs ~$13 for batched serverless GPU (3K tokens/sec, ~3 hours). Interactive idle time is expensive: Qwen3.6 35B on H200 at $4.54/hour means 10 idle hours costs ~$45.
Industry Insight
- Teams should prioritize harness engineering investment over model procurement decisions, as the same model with a better harness can dramatically outperform competitors—this is an architecture decision, not a deployment detail.
- Organizations building agent systems should match their billing model to the interaction pattern: per-token APIs for latency-sensitive interactive workflows, and serverless GPU-hour pricing for throughput-oriented batch or async workloads, with serverless typically outperforming reservations given the 5–10× peak-to-average ratios common in agentic work.
- The three-mode taxonomy (interactive, remote, async) provides a practical framework for agent architecture decisions, with each mode requiring different infrastructure choices and having distinct cost profiles that can differ by an order of magnitude at scale.
Disclaimer: The above content is generated by AI and is for reference only.