Meituan Releases LongCat-2.0: A 1.6T-Parameter Open MoE Model with Native 1M Context and LongCat Sparse Attention 美团发布LongCat-2.0：一款拥有原生1M上下文和LongCat稀疏注意力的1.6万亿参数开源MoE模型

TL;DR

• Meituan released LongCat-2.0, a 1.6T-parameter open-source Mixture-of-Experts (MoE) model with approximately 48B active parameters per token, specifically optimized for agentic coding workflows.
• The model features a native 1-million-token context window enabled by LongCat Sparse Attention (LSA) and was trained entirely on domestic AI ASIC superpods, demonstrating stable training on non-Nvidia hardware.
• Key architectural innovations include zero-computation experts for dynamic activation scaling, a 135B-parameter N-gram embedding module to reduce memory I/O, and a post-training pipeline (MOPD) fusing Agent, Reasoning, and Interaction capabilities.
• In benchmark evaluations, LongCat-2.0 achieved 59.5 on SWE-bench Pro, slightly outperforming GPT-5.5, and showed strong performance on Terminal-Bench 2.1 and multilingual repository tasks, though it trails frontier models on general agent benchmarks.
• The model is accessible via API with OpenAI and Anthropic compatibility, supporting use cases like whole-repository reasoning, multi-step terminal tasks, and cross-language migrations, with weights pending release.

Why It Matters

This release highlights the growing viability of training and serving massive models on domestic, non-Nvidia hardware, offering a potential pathway for regions or entities facing supply chain constraints. For AI practitioners, LongCat-2.0 sets a new standard for long-context agentic coding, proving that models can effectively reason over entire codebases without relying on summarization hacks. Its open nature and specific optimization for software engineering agents make it a critical resource for developers building autonomous coding tools.

Technical Details

• Architecture & Efficiency: Utilizes a Shortcut-connected MoE (ScMoE) backbone with zero-computation experts that route simple tokens (like punctuation) without heavy compute, managed by a PID controller to maintain a dynamic activation window between 33B and 56B parameters.
• Long-Context Mechanism: Implements LongCat Sparse Attention (LSA), an evolution of DeepSeek Sparse Attention, which combines Streaming-aware, Cross-Layer, and Hierarchical indexing to achieve near-linear scaling for a 1M-token context window, avoiding the memory wall associated with quadratic attention.
• Training Infrastructure: Pretrained on over 35 trillion tokens using domestic AI ASIC superpods, achieving remarkable stability with no loss spikes or rollbacks, a significant achievement given the less mature tooling ecosystem for non-Nvidia accelerators compared to CUDA.
• Performance Benchmarks: Reports 59.5 on SWE-bench Pro (beating GPT-5.5's 58.6), 70.8 on Terminal-Bench 2.1, and 77.3 on SWE-bench Multilingual, indicating specialized strength in real-world software engineering and cross-language tasks.
• Serving Optimization: Employs a 6D parallelism scheme, prefill-decode disaggregation, 'super kernels,' and L2-cache weight prefetching to minimize I/O latency, allowing efficient inference despite the model's massive scale.

Industry Insight

The successful deployment on domestic ASICs signals a strategic shift toward hardware diversification in AI infrastructure, reducing reliance on Nvidia GPUs and encouraging investment in alternative accelerator ecosystems. Developers should prioritize agentic frameworks that leverage long-context windows for codebase-wide reasoning, as this capability significantly reduces the complexity of managing fragmented context in large software projects. Finally, the open release of such a powerful coding model may accelerate the adoption of autonomous software engineering agents, forcing competitors to improve their own long-context and agentic coding benchmarks rapidly.

TL;DR

• 美团发布LongCat-2.0，这是一个拥有1.6万亿总参数、每次Token激活约480亿参数的开源MoE大模型，专为Agentic Coding设计。
• 模型原生支持100万Token上下文窗口，并采用“零计算专家”、长Cat稀疏注意力（LSA）及N-gram嵌入等技术优化推理成本与效率。
• 训练与推理完全基于国产AI ASIC超级集群完成，实现了在非英伟达硬件上的稳定大规模训练与服务部署。
• 在SWE-bench Pro等代码基准测试中表现优异，声称超越GPT-5.5，但在通用智能基准上仍落后于前沿系统。
• 目前仅通过API平台提供访问（兼容OpenAI/Anthropic接口），权重尚未开源，本地自托管暂不可用。

为什么值得看

对于AI从业者和企业而言，LongCat-2.0展示了在特定垂直领域（如软件工程代理）实现超大规模模型落地的可行路径，特别是其针对Agentic工作流的深度优化。同时，该模型在国产硬件上的成功训练与部署，为摆脱单一硬件依赖、构建自主可控的大模型基础设施提供了重要的工程参考案例。

技术解析

• 架构与参数规模：LongCat-2.0采用Mixture-of-Experts (MoE) 架构，总参数量达1.6T，但通过动态路由机制，每个Token仅激活约33B-56B参数（平均48B），显著降低了计算开销。其前身LongCat-Flash为560B参数，此次升级带来了巨大的容量提升。
• 长上下文与注意力机制：模型原生支持1M Token上下文。为解决长序列的内存墙问题，引入了LongCat Sparse Attention (LSA)，这是对DeepSeek DSA的演进，结合了流式感知索引、跨层索引和分层索引三种正交方法，将注意力复杂度从二次方降低至接近线性。
• 高效推理优化技术：
  • 零计算专家 (Zero-computation experts)：简单Token（如标点）路由至无需计算的专家，通过PID控制器动态调整专家偏差以维持激活范围平衡。
  • N-gram Embedding：集成135B参数的N-gram嵌入模块，捕捉局部密集Token关系，减少大Batch解码时的内存I/O。
  • 服务优化：采用6D并行策略、预填充-解码分离架构（Prefill-Decode Disaggregation）、Super Kernels及L2缓存权重预取，以隐藏I/O延迟。
• 训练稳定性与硬件：预训练消耗超过35万亿Token，历时数百万加速器小时，且在非Nvidia硬件上实现了无回滚、无不可恢复损失尖峰的稳定性，证明了国产ASIC集群在超大规模训练中的成熟度。
• 后训练与对齐：使用MOPD管道融合Agent、推理和交互三类教师专家组，统一模型能力。

行业启示

• 垂直领域专用化趋势：通用大模型竞争白热化背景下，面向特定工作流（如Agentic Coding）的深度优化模型更具商业落地价值。LongCat-2.0证明通过架构创新（如稀疏注意力、动态激活）可以在保持高性能的同时控制成本。
• 算力自主可控的可行性：在国产ASIC集群上成功训练并服务1.6T参数模型，标志着国内算力基础设施已具备支撑顶级大模型全生命周期的能力，这将加速非英伟达生态的大模型研发与应用。
• Agent工作流的工程化挑战：模型对终端执行、错误恢复和多文件仓库理解的支持，表明未来的AI应用将从“对话”转向“行动”。开发者需关注如何将大模型无缝集成到现有的DevOps和Agent Harness中，以发挥其最大效能。