NVIDIA releases Nemotron-Labs-TwoTower open-weight diffusion language model
Original title:NVIDIA Releases Nemotron-Labs-TwoTower: an Open-Weight Diffusion Language Model Built on a Frozen Autoregressive Nemotron-3-Nano-30B-A3B Backbone
AI summary
NVIDIA released Nemotron-Labs-TwoTower, a diffusion language model based on the frozen autoregressive backbone Nemotron-3-Nano-30B-A3B. It uses a two-tower architecture: the context tower is frozen and the denoiser tower is trained, collaborating through layer-aligned cross-attention and state seeding. Evaluated in BF16 on 2×H100, it retains 98.7% of the AR baseline quality and improves generation throughput by 2.42× (γ=0.8, block size S=16). The denoiser was trained on about 2.1T tokens, and the backbone was pretrained on 25T tokens. It has about 60B total parameters and about 3B active parameters per token per tower. It supports three decoding modes: diffusion, simulated AR and AR.
Key Takeaways
- NVIDIA released Nemotron-Labs-TwoTower,a diffusion language model based on the frozen…
- It uses a two-tower architecture: the context tower is frozen and the denoiser tower is…
- Evaluated in BF16 on 2×H100,it retains 98.
- 7% of the AR baseline quality and improves generation throughput by 2.
- 8,block size S=16).
Quick Facts
- Published
- Updated
- Source
- MarkTechPost(RSS)
- Category
- Model Releases
- AI score
- 73
NVIDIA这个TwoTower把扩散解码接在已有的AR骨干上,几乎无损质量却让吞吐翻倍,并且开源可商用,对批量文本生成的团队是实在的加速工具。
Related entities
Source & copyright
This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.
How to Cite
When citing this article, keep the title, date, and permalink, and credit Orange News as the source.
Orange News: NVIDIA releases Nemotron-Labs-TwoTower open-weight diffusion language model. July 1, 2026. https://new.nat6.net/en/news/cmr1ti4v001xlsl8zovi1yoqa/