NVIDIA releases Nemotron-Labs-3-Puzzle-75B-A9B: compressed hybrid MoE model with 2.03x higher server throughput
Original title:NVIDIA Releases Nemotron-Labs-3-Puzzle-75B-A9B: A Compressed Hybrid MoE LLM Delivering 2.03x Server Throughput at Matched User Throughput
AI summary
NVIDIA released Nemotron-Labs-3-Puzzle-75B-A9B, a compressed variant of Nemotron-3-Super, reducing total parameters from 120.7B to 75.3B and active parameters from 12.8B to 9.3B while maintaining an 88-block hybrid layout (40 Mamba, 40 MoE, 8 attention). On an 8×B200 node, in 8K/64K scenarios with matched user throughput of ≥100 tok/s, server throughput improves by 2.03x. On a single H100, 1M-token concurrency increases from 1 to 8, and weight footprint drops from 70 GB to 44.5 GB. The iterative Puzzle method scores 0.57 higher on average than the single-step approach. The cost: Arena-Hard-V2 drops 4.2 points and SWE-Bench drops 2.6 points. Hugging Face provides BF16, FP8, and NVFP4 checkpoints.
Key Takeaways
- NVIDIA released Nemotron-Labs-3-Puzzle-75B-A9B,a compressed variant of Nemotron-3-Super,reducing…
- 7B to 75.
- 3B and active parameters from 12.
- 3B while maintaining an 88-block hybrid layout (40 Mamba,40 MoE,8 attention).
- On an 8×B200 node,in 8K/64K scenarios with matched user throughput of ≥100 tok/s,server throughput…
Quick Facts
- Published
- Updated
- Source
- MarkTechPost(RSS)
- Category
- Model Releases
- AI score
- 70
把120B MoE压到75B后,同节点服务吞吐几乎翻倍,单张H100百万token并发从1涨到8。对长上下文RAG和AI编码助手这类对吞吐敏感的产品来说,这不是论文调优,是实打实的降本方案。
Related entities
Source & copyright
This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.
How to Cite
When citing this article, keep the title, date, and permalink, and credit Orange News as the source.
Orange News: NVIDIA releases Nemotron-Labs-3-Puzzle-75B-A9B: compressed hybrid MoE model with 2.03x higher server throughput. July 9, 2026. https://new.nat6.net/en/news/cmrd9x7xz01wzih4bch5uoywh/