Generate Trajectories, Reasoning Traces, and Auto-Labels with NVIDIA Alpamayo 2 Super
NVIDIA Alpamayo 2 Super is a 34B-parameter open reasoning vision-language-action (VLA) model combining a 32B Cosmos 3 Super Reasoner with a 2B diffusion-based Action Expert, post-trained via reinforcement learning The model unifies trajectory generation, Chain-of-Causation reasoning traces, meta-action prediction, VQA with 2D grounding, and auto-labeling into a single foundation model for autonomous vehicle development It supports 360-degree perception from up to seven cameras and delivers state
Analysis
TL;DR
- NVIDIA Alpamayo 2 Super is a 34B-parameter open reasoning vision-language-action (VLA) model combining a 32B Cosmos 3 Super Reasoner with a 2B diffusion-based Action Expert, post-trained via reinforcement learning
- The model unifies trajectory generation, Chain-of-Causation reasoning traces, meta-action prediction, VQA with 2D grounding, and auto-labeling into a single foundation model for autonomous vehicle development
- It supports 360-degree perception from up to seven cameras and delivers state-of-the-art results across trajectory prediction (minADE_6: 0.911 m), AV reasoning (0.433), LingoQA (79.2), meta-action IoU, VQA similarity (0.652), 2D grounding IoU (0.71), and closed-loop AlpaSim Score (1.50 ± 0.13)
- Released under the permissive OpenMDW-1.1 license, enabling commercial fine-tuning, derivative models, and unrestricted deployment of distilled variants
- The unified architecture replaces fragmented multi-model AV pipelines, allowing the same model to serve as an offline policy teacher, evaluation critic, data engine, and customization starting point
Why It Matters
Alpamayo 2 Super addresses a critical bottleneck in autonomous vehicle development: the fragmentation of separate models for trajectory prediction, reasoning, scene understanding, and data labeling, which hinders cross-task comparison, behavior interpretability, and representation reuse. By unifying these capabilities into a single open-weight model, it lowers the barrier for AV teams to adopt reasoning-capable foundation models and accelerates the iteration cycle from data curation to policy evaluation. Its permissive licensing and public inference notebooks further democratize access for researchers and practitioners who lack the resources to build equivalent systems from scratch.
Technical Details
- Architecture: 34B total parameters split into a 32B Cosmos 3 Super Reasoner (multimodal video-language interpreter) and a 2B diffusion-based Action Expert that maps internal representations to future ego-vehicle trajectories; post-trained with reinforcement learning
- Input/Output Modalities: Accepts up to seven surround-view cameras, language context, and prior motion history; outputs future trajectories, Chain-of-Causation (CoC) reasoning traces, high-level meta-actions (yield, lane change, stop), grounded VQA answers, and structured auto-labels with 2D bounding boxes
- Inference Pipeline: Trajectory sampling uses diffusion-based decoding (10 inference steps) with top-p=0.98 and temperature=0.6; CoC traces are generated jointly with trajectories to provide causal explanations for driving decisions
- Benchmarks & Results: Achieves minADE_6 of 0.911 m on trajectory prediction, 0.433 on AV reasoning, 79.2 on LingoQA, lateral/meta-action IoU of 74.59/61.91/73.55, VQA answer similarity of 0.652, 2D grounding IoU of 0.71, and a closed-loop AlpaSim Score of 1.50 ± 0.13, all reported as state-of-the-art
- Licensing & Availability: Weights hosted on Hugging Face, inference notebooks on GitHub, released under OpenMDW-1.1 (Linux Foundation permissive license) allowing commercial fine-tuning, derivative models, and redistribution without additional NVIDIA permission; model outputs carry no license restrictions
Industry Insight
- The shift toward unified reasoning VLA models signals the end of siloed AV model stacks; organizations should evaluate consolidating their trajectory, reasoning, and labeling pipelines into a single foundation model to reduce engineering overhead and improve cross-task consistency
- Chain-of-Causation reasoning traces provide an interpretable audit trail that can be integrated into safety validation workflows (e.g., NVIDIA Halos), making them valuable for regulatory compliance and failure diagnosis in production AV systems
- The permissive OpenMDW-1.1 license lowers adoption friction for commercial AV developers; teams should prioritize distilling smaller variants of Alpamayo 2 Super for edge deployment while retaining the full model for offline policy teaching and data curation
Disclaimer: The above content is generated by AI and is for reference only.