Model Releases

NVIDIA releases Nemotron-Labs-3-Puzzle-75B-A9B: compressed hybrid MoE model with 2.03x higher server throughput

Original title:NVIDIA Releases Nemotron-Labs-3-Puzzle-75B-A9B: A Compressed Hybrid MoE LLM Delivering 2.03x Server Throughput at Matched User Throughput

AI summary

NVIDIA released Nemotron-Labs-3-Puzzle-75B-A9B, a compressed variant of Nemotron-3-Super, reducing total parameters from 120.7B to 75.3B and active parameters from 12.8B to 9.3B while maintaining an 88-block hybrid layout (40 Mamba, 40 MoE, 8 attention). On an 8×B200 node, in 8K/64K scenarios with matched user throughput of ≥100 tok/s, server throughput improves by 2.03x. On a single H100, 1M-token concurrency increases from 1 to 8, and weight footprint drops from 70 GB to 44.5 GB. The iterative Puzzle method scores 0.57 higher on average than the single-step approach. The cost: Arena-Hard-V2 drops 4.2 points and SWE-Bench drops 2.6 points. Hugging Face provides BF16, FP8, and NVFP4 checkpoints.

Key Takeaways

  • NVIDIA released Nemotron-Labs-3-Puzzle-75B-A9B,a compressed variant of Nemotron-3-Super,reducing…
  • 7B to 75.
  • 3B and active parameters from 12.
  • 3B while maintaining an 88-block hybrid layout (40 Mamba,40 MoE,8 attention).
  • On an 8×B200 node,in 8K/64K scenarios with matched user throughput of ≥100 tok/s,server throughput…

Quick Facts

Published
Updated
Source
MarkTechPost(RSS)
Category
Model Releases
AI score
70
Read the original Read the full article at the original source
Why it matters

把120B MoE压到75B后,同节点服务吞吐几乎翻倍,单张H100百万token并发从1涨到8。对长上下文RAG和AI编码助手这类对吞吐敏感的产品来说,这不是论文调优,是实打实的降本方案。

Related entities

Source & copyright

This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.

Original article · AIHOT entry

How to Cite

When citing this article, keep the title, date, and permalink, and credit Orange News as the source.

Orange News: NVIDIA releases Nemotron-Labs-3-Puzzle-75B-A9B: compressed hybrid MoE model with 2.03x higher server throughput. July 9, 2026. https://new.nat6.net/en/news/cmrd9x7xz01wzih4bch5uoywh/

View all →