Z.ai Releases GLM-5.3-Flash: A 320B-A18B Natively Multimodal MoE With a 1M-Token Context
Z.ai released GLM-5.3-Flash, a 320B-parameter MoE model with 18B active parameters per token, featuring native multimodal (image/video) support and a 1M-token context window, licensed under MIT with weights on Hugging Face. The architecture introduces hybrid attention (KDA linear + NoPE sparse MLA), IndexPool for KV cache compression, and Manifold-Constrained Hyper-Connections, achieving ~3× less attention compute and 4.4× smaller KV cache versus GLM-5.3. Benchmark results are strong: 84.3 on Te
Analysis
TL;DR
- Z.ai released GLM-5.3-Flash, a 320B-parameter MoE model with 18B active parameters per token, featuring native multimodal (image/video) support and a 1M-token context window, licensed under MIT with weights on Hugging Face.
- The architecture introduces hybrid attention (KDA linear + NoPE sparse MLA), IndexPool for KV cache compression, and Manifold-Constrained Hyper-Connections, achieving ~3× less attention compute and 4.4× smaller KV cache versus GLM-5.3.
- Benchmark results are strong: 84.3 on Terminal-Bench 2.1 (near Claude Opus 4.8's 85.0), 63.4 on DeepSWE v1.1 (up from GLM-5.2's 46.2), and 29.0 on Z.ai Code Bench (within 0.5 of Opus 4.8).
- Priced at $0.15/M input and $0.50/M output, it delivers roughly one-tenth the cost of GLM-5.2 while offering 3× the usable quota across all GLM Coding Plan tiers.
- Self-hosting requires ~306 GiB FP8 weights and NVIDIA Hopper or newer GPUs; the preview was served anonymously as "Ox Alpha" on domestically produced Chinese AI chips with a 3× end-to-end serving improvement.
Why It Matters
GLM-5.3-Flash represents a significant step toward cost-effective, high-capability multimodal coding models, bridging the gap between premium offerings like Claude Opus 4.8 and budget-conscious deployments. Its MIT license and open weights lower barriers for researchers and enterprises to experiment with and fine-tune a model that competes at the top tier while costing a fraction of the price. The hybrid attention and IndexPool innovations also provide a blueprint for scaling to million-token contexts without prohibitive memory costs.
Technical Details
- Architecture: 45-layer MoE with 8 of 288 experts activated per token, native FP8 weights, and one MTP draft layer. Combines KDA linear-attention layers for local dependencies with NoPE sparse MLA layers for global context retrieval.
- IndexPool: Compresses groups of indexer key vectors through weighted pooling, reducing attention compute by ~3× and KV cache size by 4.4× compared to GLM-5.3, critical for million-token context handling.
- mHC (Manifold-Constrained Hyper-Connections): Improves scaling efficiency, roughly halving activated parameters and layer count versus GLM-4.5 at similar total parameter counts.
- Training: Base model trained on a 30T-token multimodal corpus, with image and video input capabilities natively integrated.
- Serving: Supports self-hosting via vLLM (NVIDIA Hopper+), SGLang, TokenSpeed, and KTransformers. The Ox Alpha preview ran on domestic Chinese AI chips using a custom SGLang-based engine with disaggregated encoding, prefill, and decoding.
Industry Insight
- Cost-performance shift: At one-tenth the price of its predecessor with near-Opus-level coding benchmarks, GLM-5.3-Flash could accelerate adoption of agentic AI workflows in software development, IT automation, and enterprise back-office operations where cost per task is a decisive factor.
- Self-hosting门槛: The 306 GiB FP8 weight requirement and Hopper+ GPU dependency mean only well-capitalized organizations can self-host; most will rely on the API, making pricing strategy and quota tiers (3× GLM-5.3) a key competitive lever for Z.ai.
- Multimodal coding gap: While coding benchmarks are strong, vision performance trails Gemini 3.7 Flash on BabyVision and MVbench, suggesting that teams requiring heavy image/video reasoning should evaluate use-case-specific trade-offs before adopting.
Disclaimer: The above content is generated by AI and is for reference only.