tencent/Hy3 腾讯/Hy3

TL;DR

• Tencent released Hy3, a 295B-parameter Mixture-of-Experts (MoE) model with 21B active parameters, licensed under Apache 2.0.
• The model features a 3.8B MTP layer and supports a 256K context window, offering significant efficiency gains through sparse activation.
• Hy3 outperforms similarly sized models and rivals flagship open-source competitors with 2-5x more parameters, demonstrating strong utility in productivity tasks.
• Post-training was scaled up using high-quality data and feedback from over 50 products following the Hy3 Preview launch.
• The model is available on Hugging Face in full size (598GB) and FP8 quantized (300GB), with temporary free access on OpenRouter.

Why It Matters

This release highlights the continued trend of leveraging Mixture-of-Experts architectures to achieve high performance with reduced inference costs, making large-scale models more accessible. For AI practitioners, Hy3 offers a competitive, commercially viable alternative to proprietary flagship models, particularly for applications requiring long-context understanding and complex reasoning. The Apache 2.0 license further encourages broad adoption and integration into enterprise workflows without restrictive usage terms.

Technical Details

• Architecture: 295B total parameters using Mixture-of-Experts (MoE) design, with only 21B active parameters per token and an additional 3.8B MTP (Multi-Token Prediction) layer parameters.
• Context & Quantization: Supports a 256K context length; available in full precision (598GB) and FP8 quantized (300GB) formats.
• Training Strategy: Utilized high-quality data scaling and incorporated feedback from 50+ products during post-training to enhance utility and productivity task performance.
• Performance: Claims to outperform similar-size models and compete with larger open-source flagships (2-5x parameter count) in benchmark evaluations.

Industry Insight

The availability of a high-performance, Apache 2.0 licensed MoE model with such a large context window signals a shift toward more efficient, cost-effective large language models for enterprise deployment. Developers should evaluate Hy3 for workloads requiring long-document processing or complex multi-step reasoning, as its sparse activation structure may offer better latency and throughput compared to dense models of equivalent capability. Additionally, the temporary free access on OpenRouter provides an opportunity for rapid prototyping and benchmarking against existing solutions before committing to infrastructure changes.

TL;DR

• 腾讯发布开源模型 Hy3，采用 MoE 架构，总参数量 295B，激活参数 21B，支持 256K 上下文长度。
• 模型在性能上超越同规模模型，并媲美参数量大 2-5 倍的旗舰开源模型，显著提升生产力任务表现。
• 提供 Hugging Face 上的全尺寸（598GB）及 FP8 量化版本（300GB），并在 OpenRouter 限时免费开放。

为什么值得看

Hy3 展示了通过高质量后训练数据和 MoE 架构优化，以较低激活参数实现顶级性能的技术路径，为开源社区提供了极具竞争力的高性能基座。其开放的许可协议和量化版本降低了部署门槛，有助于推动企业级 AI 应用的落地与效率提升。

技术解析

• 架构与规模：Hy3 是腾讯 Hy Team 开发的 MoE 模型，总参数量达 295B，其中 21B 为激活参数，包含 3.8B 的 MTP 层参数，实现了计算效率与性能的平衡。
• 训练优化：基于 Hy3 Preview 版本的反馈，团队收集了来自 50 多个产品的数据，并使用更高质量的数据进行了扩展后训练（Post-training），从而提升了模型在各类产品中的实用性。
• 部署规格：模型上下文窗口长度为 256K。在 Hugging Face 上提供两种格式：全精度版本约 598GB，FP8 量化版本约 300GB，兼顾了高性能推理与资源受限场景的需求。

行业启示

• MoE 架构的主流化：Hy3 的成功进一步验证了 MoE 架构在平衡大规模模型性能与推理成本方面的优势，未来更多高性能模型将倾向于采用稀疏激活策略。
• 数据质量重于数量：通过针对性的高质量后训练显著超越更大参数量的模型，表明在算力边际效应递减的背景下，数据清洗与合成质量将成为模型迭代的核心竞争力。
• 开源生态的商业闭环：腾讯以 Apache 2.0 协议开源并配合限时免费 API，旨在通过开发者社区反馈反哺模型迭代，这种“开源引流+商业变现”的模式将成为大模型厂商的标准动作。