Death of Params: Z.ai CEO Jie Tang on GLM 5.3 and the new Post-training Scaling Law
Prof Jie Tang argues parameter count alone is insufficient for scaling; four additional knobs matter: data volume, compute allocation, deployment conditions, and MoE sparsity (XA-YB notation) GLM-5.3's major improvements stem from RL on long-horizon, production-realistic environments where models take end-to-end ownership of multi-day engineering tasks A synthetic environment pipeline uses research agents to collect real work patterns, judge agents to verify solvability, and verifiers trained wi
Analysis
TL;DR
- Prof Jie Tang argues parameter count alone is insufficient for scaling; four additional knobs matter: data volume, compute allocation, deployment conditions, and MoE sparsity (XA-YB notation)
- GLM-5.3's major improvements stem from RL on long-horizon, production-realistic environments where models take end-to-end ownership of multi-day engineering tasks
- A synthetic environment pipeline uses research agents to collect real work patterns, judge agents to verify solvability, and verifiers trained without reference solutions to produce reliable binary rewards
- Ornith-1.5 emerges as a significant open-weight family (9B dense, 35B MoE, 397B MoE) under MIT license, claiming end-to-end self-improvement with strong agentic/coding evals
- Quantization advances reach 1-bit with ~77% BF16 accuracy retention on 8GB RAM, while Dynamic V3 compression claims 10% accuracy gains at equal size
Why It Matters
The shift from parameter-count-centric scaling to multi-knob optimization fundamentally changes how practitioners should think about model development and deployment. The emergence of synthetic, self-improving training environments signals that post-training scaling may soon be limited by environment quality rather than model architecture, opening new pathways for open-weight models to close gaps with closed systems.
Technical Details
- GLM-5.3 training uses long-horizon RL environments mirroring real production workflows, including ML infrastructure tasks where models diagnose bottlenecks, implement optimizations, and deliver measurable speedups across compute clusters, storage, and codebases
- The synthetic environment pipeline operates fully autonomously: research agents extract task patterns from real work into runnable multi-step environments with hidden state; judge agents verify solvability; verifiers pass oracle, no-op, and unsolved-state checks without access to reference solutions
- Ornith-1.5 is released in three variants (9B dense, 35B MoE, 397B MoE) under MIT license with FP8, GGUF, MLX, and NVFP4 quantization support, achieving Terminal-Bench 2.1: 86.1, SWE-Bench Verified: 86, DeepSWE: 56, HLE: 44.6, Tool Decathlon: 71.2
- Unsloth AI's Dynamic V3 compression on Qwen3.8-27B GGUF claims ~10% accuracy improvement at equal size, while their 1-bit quantization retains ~77% of BF16 accuracy running on 8GB RAM, evaluated via the new Divergence-300 metric
- Prof Jie Tang's XA-YB notation formalizes MoE sparsity as a scaling knob alongside data, compute, and deployment conditions, challenging Chinchilla-era assumptions of fixed token-per-parameter ratios
Industry Insight
The industry is moving toward environment-driven scaling where synthetic task generation and verification become the bottleneck rather than model size, creating opportunities for organizations that can build high-quality long-horizon training environments. Open-weight models are rapidly closing the capability gap through self-improvement loops and aggressive quantization, suggesting that deployment efficiency and customization will become key differentiators over raw parameter counts. Practitioners should prioritize evaluating models on long-chain reasoning benchmarks and agentic workloads rather than relying on traditional metric shorthand.
Disclaimer: The above content is generated by AI and is for reference only.