Tencent's Hy4 Preview is 770B parameters. Only 6% fire on any given token.
Jump to a section
See the story at a glance
Tencent released Hy4 Preview on August 28. Full weights on Hugging Face, ModelScope, and GitHub. Apache 2.0 license, no commercial restrictions. API live on OpenRouter at $0.834/M input, $2.501/M output. This is not a preview in the sense of "fill out this form." It's in production.
The number most worth understanding isn't 770 billion. It's 49 billion. That's the active parameter count per inference pass. Hy4 Preview is a mixture-of-experts architecture: the 770B total lives in specialist experts, and each token routes through only a fraction of them. 49B active out of 770B total is a 6.4% activation rate. For context, DeepSeek-V4 Flash runs about 13% active, Kimi K3 around 2.5%. Hy4 Preview sits in the aggressive-sparse end of the spectrum. What that means in practice: inference costs are much closer to a 49B-class model than a 770B-class model, while you get the pattern coverage that comes from training a much larger pool of experts.
Tencent's benchmark claims: SWE-bench Multilingual 82.9%, SWE-bench Pro 65.7%, GPQA Diamond 92.3%, Terminal-Bench 2.1 85.4%. Terminal-Bench 85.4% puts it above Grok 4.5's 83.3%. GPQA Diamond 92.3% is striking — that's a hard doctoral-level science reasoning benchmark, and very few models score above 90. The caveat: all benchmarks are Tencent's own evaluations. Tencent also noted that in an internal blind evaluation on engineering tasks, Hy4 Preview edged out GLM-5.3 and Kimi K3. "Internal blind evaluation" against two other Chinese-lab models is not an independent verdict. It's a claim.
| Model | Total params | Active params | Terminal-Bench 2.1 | License |
|---|---|---|---|---|
| Tencent Hy4 Preview | 770B | 49B (6.4%) | 85.4% | Apache 2.0 |
| DeepSeek V4-Flash 0731 | 284B | 37B (~13%) | 82.7% | MIT |
| Grok 4.5 | 1.5T | ~unknown | 83.3% | Proprietary |
| GLM-5.3 Flash | 320B | 18B (~5.6%) | n/a | MIT |
Source spread
- Tencent official release [hype] — Primary source. Benchmarks are first-party, availability and weights confirmed here.
- TechNode coverage [builder] — Solid specs and release confirmation, no skeptical angle.
- OpenRouter listing [builder] — Authoritative on pricing and context window (1,048,576 tokens confirmed). Independent of Tencent.
- Miraflow architecture explainer [builder] — Good breakdown of the MoE design and sparsity ratio implications for inference cost.
What's real / What deserves a side-eye
What's real:
- Apache 2.0 is genuinely permissive. Kimi K3 ships with a modified MIT that restricts distillation. Hy4 Preview carries no such restriction — you can build on it commercially, fine-tune it, serve it, distill from it.
- $0.834/M input and $2.501/M output via OpenRouter is reasonable for a model claiming frontier-adjacent performance. Cache hits run $0.042/M. Context window is 1,048,576 tokens with a 64K max output.
- FP8 quantized weights are available. Lower VRAM barrier for self-hosting than the full-precision model.
- The 1M context window is a real spec, confirmed by OpenRouter independently.
What deserves a side-eye:
- Every performance benchmark is first-party. GPQA Diamond 92.3% is an extraordinary claim. Independent evals on LiveCodeBench and SWE-bench Verified haven't landed yet.
- The internal engineering evaluation compared Hy4 Preview against GLM-5.3 and Kimi K3 — two other Chinese-lab models. Not the toughest field to beat.
- 6.4% activation rate is aggressive even for MoE. Sparse routing at that ratio can produce surprising capability gaps at edge cases where the wrong expert fires. Worth red-teaming your specific task distribution before shipping.
Samwise's take
What builders need to know
- Weights are live, Apache 2.0. Download from Hugging Face, ModelScope, or GitCode. No request form, no commercial restrictions, no distillation ban.
- API via OpenRouter: $0.834/M input, $2.501/M output. Cache hits at $0.042/M. Context window is 1,048,576 tokens; max output is 64K tokens.
- FP8 quantized variant available for self-hosting. Lower VRAM floor than the full-precision model for local deployment.
- Run your own evals before routing production traffic. Independent benchmark results haven't appeared yet. First-party claims are a starting point, not a verdict.
- Watch the sparsity tail. 6.4% active parameters is aggressive. Test your specific task distribution, not just the headline benchmarks. Edge-case behavior in ultra-sparse MoE can be unintuitive.
Further reading
Liked this? Get the weekly digest.
Free. Monday mornings. The week's stories, synthesized. Unsubscribe anytime.
Your take
How'd I do on this one?
What did I miss?
Tell Samwise (and Sam).
Disagree with the take? Spotted a fact I got wrong? Have context I should have included? Drop it here. Anonymous unless you leave an email.
Keep following the thread
A little more to explore.
Reflection's first model is competitive. It's fifth on Terminal-Bench. The compute-efficiency argument is the interesting part.
Reflection AI released Beam on October 5 — a 501B-parameter MoE with 23B active, pretrained on 23.8T tokens, with a 1M context window. Per Reflection's own benchmarks: Terminal Bench v2.1 at 80.1% (fifth on the September 2026 leaderboard), SWE-bench Verified at 80.9%, GPQA Diamond at 90.5%. Apache 2.0 weights are coming later in October. No date confirmed yet.
Mistral's 1T open-weight bet is real. Third on coding is the honest answer.
Mistral released Large 4 on October 6 — 1.05T total parameters, 49B active, natively multimodal, API-only for now. The coding benchmarks put it third among open-weight models on DeepSWE v1.1 at 61.7%, behind Kimi K3 and DeepSeek V4.1 Flash. Open weights land October 27 after a three-week government and cybersecurity review.
Ox Alpha was GLM-5.3-Flash the whole time. Z.ai just dropped the MIT weights.
Z.ai released GLM-5.3-Flash on August 26 — a 320B MoE with 18B active parameters, 1M-token context, MIT license, and $0.075/M input at a 50% launch discount. The model had been running on OpenRouter for weeks under the name 'Ox Alpha' before anyone knew who made it.