AI Practices AI实践 15h ago Updated 14h ago 更新于 14小时前 55

NVIDIA AVO Reaches 100% on ARC-AGI-3, Demonstrating a Frontier-Level General-Purpose Architecture for Long-Horizon Autonomous Agents NVIDIA AVO在ARC-AGI-3上达到100%,展示面向长程自主智能体的前沿通用架构

NVIDIA's Agentic Variation Operators (AVO) is a general-purpose agent architecture that elevates Claude Opus 5 from a 30% model baseline to 100% on ARC-AGI-3, demonstrating that system design—not just model capability—drives frontier-level performance AVO achieved up to 10.5% better performance than FlashAttention-4 on GPU-kernel optimization tasks across 500+ exploration directions over seven days of autonomous operation on DGX B200 systems The architecture integrates persistent memory and a su NVIDIA AVO架构在ARC-AGI-3基准测试达到100% RHAE分数,完成全部183个关卡(25个环境),比VISTA少用12%环境操作 系统架构设计可将Claude Opus 5基线从30%提升至100%,证明系统级设计比模型能力更能决定agent性能上限 GPU内核优化中AVO连续自主运行7天,探索500+方向、提交40个版本,在DGX B200上超越FlashAttention-4达10.5% AVO通过持久化记忆和supervisor监督机制实现长周期自主运行,无需人工干预即可维持 productive engineering loop

78
Hot 热度
75
Quality 质量
82
Impact 影响力

Analysis 深度分析

TL;DR

  • NVIDIA's Agentic Variation Operators (AVO) is a general-purpose agent architecture that elevates Claude Opus 5 from a 30% model baseline to 100% on ARC-AGI-3, demonstrating that system design—not just model capability—drives frontier-level performance
  • AVO achieved up to 10.5% better performance than FlashAttention-4 on GPU-kernel optimization tasks across 500+ exploration directions over seven days of autonomous operation on DGX B200 systems
  • The architecture integrates persistent memory and a supervisor mechanism to sustain long-horizon autonomous work, allowing the agent to resume from accumulated state rather than reconstructing progress each iteration
  • The same general-purpose agent was adapted to two vastly different domains—GPU kernel optimization and interactive reasoning—by changing only environment-specific tools and evaluation, not the underlying architecture
  • AVO completed all 183 ARC-AGI-3 levels across 25 environments with 12% fewer environment actions than VISTA, achieving a perfect 100.00 RHAE score

Why It Matters

This work represents a significant shift in how the AI community evaluates and builds autonomous agents: it proves that system-level architecture can dramatically amplify frontier model capabilities, closing the gap between what a model can do in isolation and what it can accomplish in sustained, real-world tasks. For practitioners, it underscores that investing in agent harness design—persistent memory, supervision, tool integration, and failure recovery—is as critical as selecting or fine-tuning the underlying language model.

Technical Details

  • Architecture: AVO is a general-purpose coding agent system built around iterative cycles of context inspection, planning, implementation, and evaluation, supported by persistent memory that carries forward prior implementations, evaluation results, compiler/profiler outputs, and accumulated reasoning across iterations
  • Supervisor mechanism: A separate supervisor component monitors the broader search trajectory for stagnation or unproductive cycles and can redirect the main agent toward alternative strategies when progress plateaus, as demonstrated during the seven-day attention-kernel optimization run
  • GPU-kernel optimization results: AVO explored over 500 optimization directions, committed 40 kernel versions, and achieved up to 3.5% better performance than cuDNN and up to 10.5% better than FlashAttention-4 on multihead attention kernels across NVIDIA DGX B200 systems; it also adapted the evolved kernel to grouped-query attention in approximately 30 minutes of additional autonomous work
  • ARC-AGI-3 performance: AVO achieved a perfect 100.00 RHAE score, completing all 183 levels across 25 environments with 12% fewer environment actions than VISTA, using the same underlying agent architecture with only environment-specific tools and evaluation modified
  • Model baseline: The system elevates Claude Opus 5 from a 30% model-only baseline to 100% on ARC-AGI-3, illustrating the multiplicative effect of the agent harness on raw model capability

Industry Insight

  • The AVO results reinforce a strategic pivot: organizations should prioritize building robust agent harnesses with persistent memory, supervision, and recovery mechanisms rather than treating frontier models as standalone solutions; the marginal gains from system design can exceed those from incremental model improvements
  • The cross-domain adaptability of AVO—successfully applied to both low-level GPU kernel optimization and high-level interactive reasoning—suggests that general-purpose agent architectures are viable, reducing the need for domain-specific agent redesign and accelerating deployment across diverse engineering workflows
  • The emphasis on a "trusted agent stack" that integrates performance, reliability, and security at the system level signals an industry-wide shift toward holistic agent security and operational integrity, prompting practitioners to evaluate agent systems end-to-end rather than assessing model capability in isolation

TL;DR

  • NVIDIA AVO架构在ARC-AGI-3基准测试达到100% RHAE分数,完成全部183个关卡(25个环境),比VISTA少用12%环境操作
  • 系统架构设计可将Claude Opus 5基线从30%提升至100%,证明系统级设计比模型能力更能决定agent性能上限
  • GPU内核优化中AVO连续自主运行7天,探索500+方向、提交40个版本,在DGX B200上超越FlashAttention-4达10.5%
  • AVO通过持久化记忆和supervisor监督机制实现长周期自主运行,无需人工干预即可维持 productive engineering loop

为什么值得看

本文首次展示了通用agent架构在推理基准和工程优化两个截然不同的领域同时达到前沿水平,证明"系统架构设计"是解锁长周期agent性能的关键杠杆。对AI从业者而言,这重新定义了agent评估范式——模型能力只是组件之一,harness设计才是决定agent能否可靠执行多步任务的根本。

技术解析

  • 双层架构设计:主agent负责迭代循环(检查上下文→规划→实现变更→评估结果),supervisor监控全局搜索轨迹,在停滞或陷入无效循环时介入重定向策略。
  • 持久化记忆机制:保留历史实现、评估结果、编译器和profiler输出及累积推理,使agent能从当前状态恢复而非重复重建搜索过程。
  • GPU内核优化成果:在attention-kernel研究中,AVO自主探索500+优化方向、提交40个kernel版本,在NVIDIA DGX B200上multihead attention kernel性能超越cuDNN 3.5%、FlashAttention-4达10.5%,并在约30分钟内自适应扩展到grouped-query attention。
  • ARC-AGI-3适配:同一通用agent连接不同任务接口(环境特定工具和评估),在无需指令、规则或目标的未知环境中完成全部183个关卡。
  • 基线提升幅度:将Claude Opus 5从30%模型基线提升至100%系统性能,凸显harness设计对模型潜力的放大效应。

行业启示

  • 评估范式转变:评估agent系统必须区分模型能力与系统架构的贡献,单纯测试模型基线会严重低估优秀agent harness的潜力。
  • 可信agent栈需系统设计:性能、可靠性和安全性必须贯穿整个agent stack设计,不能仅依赖模型本身属性;安全与信任是系统工程问题。
  • 通用架构的可迁移性:同一AVO架构可适配从高性能计算到通用推理的跨领域任务,只需更换环境特定工具和评估接口,为构建真正通用的long-horizon agent提供了可行路径。

Disclaimer: The above content is generated by AI and is for reference only. 免责声明:以上内容由 AI 生成,仅供参考。

Agent Agent Benchmark 基准测试 Research 科学研究 LLM 大模型 GPU GPU