Tencent Releases Hy3: An Open 295B Mixture-of-Experts (MoE) Model with 21B Active Parameters and 256K Context
Tencent releases Hy3, an open-source 295B-parameter Mixture-of-Experts (MoE) model licensed under Apache 2.0, activating only 21B parameters per token for efficient inference. The architecture integrates a Multi-Token Prediction (MTP) layer and supports 256K context length, optimized for reasoning, agentic workflows, and long-context tasks. Hy3 demonstrates strong performance in STEM and coding benchmarks, with significant improvements in production reliability, including reduced hallucination r
Analysis
TL;DR
- Tencent releases Hy3, an open-source 295B-parameter Mixture-of-Experts (MoE) model licensed under Apache 2.0, activating only 21B parameters per token for efficient inference.
- The architecture integrates a Multi-Token Prediction (MTP) layer and supports 256K context length, optimized for reasoning, agentic workflows, and long-context tasks.
- Hy3 demonstrates strong performance in STEM and coding benchmarks, with significant improvements in production reliability, including reduced hallucination rates and stable tool calling across various agent scaffolds.
- It offers a cost-effective alternative to larger models like GLM-5.2 by trading some absolute coding accuracy for a significantly smaller active parameter footprint, facilitating cheaper self-hosting.
Why It Matters
Hy3 represents a strategic shift toward efficient, open-weight models capable of handling complex agentic workflows without the prohibitive costs associated with massive dense or large-active-parameter MoE models. By providing robust tool-calling stability and anti-hallucination features out of the box, it addresses critical pain points for developers building autonomous agents. The Apache 2.0 license further democratizes access to high-performance reasoning capabilities, allowing enterprises to deploy sophisticated AI solutions on their own infrastructure.
Technical Details
- Architecture: Sparse MoE with 192 experts and top-8 routing, resulting in 21B activated parameters per token. Includes a 3.8B parameter Multi-Token Prediction (MTP) layer for accelerated decoding via speculative decoding in vLLM and SGLang.
- Specifications: 80 layers (excluding MTP), 64 attention heads with GQA (8 KV heads), hidden size of 4096, and a vocabulary size of 120,832. Supports BF16 precision, with a separate FP8 checkpoint available to reduce memory footprint.
- Performance & Benchmarks: Achieves 78.0 on SWE-Bench Verified and 90.4 on GPQA Diamond. Internal evaluations show hallucination rates dropped from 12.5% to 5.4%, and multi-turn intent tracking issues decreased from 17.4% to 7.9%.
- Deployment: Requires approximately 8 GPUs (e.g., H20-3e) for full BF16 serving. Supports an OpenAI-compatible API with a
reasoning_effortflag to control chain-of-thought depth ("no_think", "low", "high").
Industry Insight
- Cost-Efficient Agentic AI: Organizations should evaluate Hy3 for self-hosted agent deployments where GPU memory costs are a constraint. The 21B active parameter count allows for competitive performance at a fraction of the hardware cost required by models with 40B+ active parameters.
- Reliability Over Raw Scores: For production environments, prioritize models with demonstrated stability in tool calling and hallucination reduction. Hy3's focus on reducing infinite loops and improving grounding suggests a trend where operational reliability is becoming as important as benchmark scores.
- Speculative Decoding Adoption: The integration of MTP and support for speculative decoding in major inference engines highlights the importance of optimizing inference speed. Developers should leverage these features to reduce latency in long-context or multi-step reasoning tasks.
Disclaimer: The above content is generated by AI and is for reference only.