Six Agent Harness Capabilities for Higher Model Performance 六种代理功能提升模型性能

TL;DR

• NOOA is an open-source, object-oriented agent framework where agents are single Python classes with methods as capabilities, fields as state, and docstrings as prompts, using type annotations as enforced contracts.
• It introduces six key interface ideas: typed input/output, pass-by-reference, code as action, programmable loop engineering, explicit object state, and model-callable harness APIs.
• NOOA features a long-term memory subsystem that stores knowledge in a human-readable SQLite database, enabling persistent, typed, relational memory without automatic summarization pipelines.
• Across benchmarks (SWE-bench Verified, CyberGym L1, ARC-AGI-3), NOOA achieves state-of-the-art performance with higher accuracy and lower token cost compared to prior harnesses.
• Agent development becomes traditional software development—agents can be diffed, tested, versioned, and refactored using standard tools.

Why It Matters

This article highlights that harness architecture—not just model choice—is critical for AI agent performance, with design choices causing double-digit swings in benchmark results and significant differences in token cost. For practitioners and researchers, NOOA provides a reproducible, inspectable, and extensible framework that bridges the gap between LLMs and structured software engineering, enabling more reliable, maintainable, and efficient agent systems. Its open release by NVIDIA Labs invites community adoption and innovation, setting a new standard for how agents should be designed and evaluated.

Technical Details

• Agent Structure: Agents are defined as single Python subclasses of Agent, where methods represent capabilities, fields represent state, docstrings serve as prompts, and type annotations act as runtime-enforced contracts. Methods marked with ellipses (...) are completed at runtime via LLM-driven loops.
• Six Interface Ideas:
  1. Typed Input/Output: All agentic calls use validated typed arguments and return values instead of free text.
  2. Pass by Reference: Models operate on live Python objects, receiving bounded previews rather than serialized dumps.
  3. Code as Action: The model acts by generating executable Python code with control flow and inline method calls.
  4. Programmable Loop Engineering: Orchestration loops are ordinary Python, writable by developers or the model itself.
  5. Explicit Object State: Durable, typed state resides on the agent object, not just in conversation history.
  6. Model-Callable Harness APIs: Context blocks and event history are exposed as APIs the model can inspect and manage.
• Memory System: Long-term memory is stored in a human-readable SQLite database. Records include types, importance tags, and relationships (e.g., “supports”, “contradicts”). A background reflection pass merges duplicates, links related records, distills insights, and prunes obsolete data. Memories can reference live agent state and are shared across sessions without retraining.
• Benchmarks & Performance:
  • SWE-bench Verified: Achieves 82.2% success rate with GPT-5.5 (above SOTA of 79.2%) and 79.8% with Opus 4.6 using a general-purpose 253-line agent.
  • CyberGym L1: Solves 86.8% with GPT-5.5, outperforming most closed-source systems; runs with network access blocked and rule-based validation to ensure reasoning from code alone.
  • ARC-AGI-3: Demonstrates improved reasoning capability through memory accumulation (+11.8 RHAE points over file-based notes).
• Open Release: Code, data, evaluations, and benchmark agents are publicly available for reproducibility, extension, and community challenge.

Industry Insight

The NOOA framework signals a shift toward treating AI agents as first-class software artifacts—emphasizing modularity, testability, and maintainability over monolithic prompt-engineering approaches. Practitioners should prioritize harness design as a core component of agent development, leveraging typed interfaces and persistent memory to improve reliability and reduce token costs. As open-source frameworks like NOOA mature, we can expect increased standardization in agent architecture, faster iteration cycles, and broader adoption of LLM-powered agents in production environments where transparency and auditability are essential.

TL;DR

• NOOA是NVIDIA Labs推出的开源面向对象智能体框架，将智能体定义为单个Python类，通过方法、字段和docstring整合能力、状态与提示。
• 支持长时记忆持久化与知识图谱构建，基于SQLite存储实现跨会话知识积累，无需重训练。
• 在SWE-bench Verified、CyberGym L1和ARC-AGI-3基准上取得SOTA性能，显著降低token成本并提升准确率。
• 强调“智能体架构”对模型表现的影响不亚于模型本身， harness设计可带来双位数结果波动。
• 提供完整代码、数据与评估流程，支持社区复现、挑战与扩展。

为什么值得看

该文章揭示了智能体性能的核心瓶颈往往不在于大模型本身，而在于其外围架构（harness）的设计方式。NOOA以软件工程化的方式重构智能体开发流程，为AI从业者提供了可测试、可调试、可协作的智能体构建范式，具有极高的工程落地价值与研究参考意义。

技术解析

• 面向对象智能体建模：NOOA将智能体封装为单一Python类，方法代表能力，字段表示状态，docstring作为提示词，类型注解作为运行时契约；带省略号(...)的方法由LLM驱动循环在运行时补全逻辑，其余方法按标准Python执行。
• 六项核心接口机制：包括 typed input/output（强类型输入输出）、pass by reference（引用传递对象而非序列化文本）、code as action（模型直接生成Python代码）、programmable loop engineering（编排逻辑为普通Python）、explicit object state（状态持久于对象实例）、model-callable harness APIs（模型可调用上下文与事件管理API）。
• 自主记忆管理系统：智能体通过可调用的工具主动写入、查询和修正记忆记录，每条记录含类型、重要性标签及关系链接（如“supports”, “contradicts”），形成结构化知识图谱；后台reflection pass自动合并冗余、提炼洞察、修剪过期信息，所有数据存于人类可读的SQLite文件。
• 跨任务通用性与高效性：使用仅253行的通用agent在SWE-bench Verified上达82.2%成功率（超越当时榜单SOTA 79.2%），在CyberGym L1解决86.8%问题且禁用网络访问，证明其泛化能力强、推理深度高；ARC-AGI-3中引入记忆模块后RHAE指标提升+11.8点。
• 开放可复现生态：完整发布框架源码、内存系统、能力测试套件、基准代理及评估脚本，确保结果透明、流程可控，便于第三方验证与二次开发。

行业启示

• 智能体竞争力将从“模型选型竞争”转向“架构设计竞争”，企业应重视harness工程化建设，将其视为提升效率与控制力的关键杠杆。
• 长期记忆与知识图谱集成将成为主流智能体标配，建议优先采用结构化存储方案（如SQLite+关系建模）替代纯文本日志或云端摘要服务，以保障隐私、可审计性与低延迟。
• 推动智能体开发纳入传统软件生命周期管理（版本控制、单元测试、代码审查），不仅能提高可靠性，还能赋能AI辅助编程工具参与智能体迭代，形成人机协同开发闭环。