Model Releases

NVIDIA releases Nemotron-Labs-TwoTower open-weight diffusion language model

Original title:NVIDIA Releases Nemotron-Labs-TwoTower: an Open-Weight Diffusion Language Model Built on a Frozen Autoregressive Nemotron-3-Nano-30B-A3B Backbone

AI summary

NVIDIA released Nemotron-Labs-TwoTower, a diffusion language model based on the frozen autoregressive backbone Nemotron-3-Nano-30B-A3B. It uses a two-tower architecture: the context tower is frozen and the denoiser tower is trained, collaborating through layer-aligned cross-attention and state seeding. Evaluated in BF16 on 2×H100, it retains 98.7% of the AR baseline quality and improves generation throughput by 2.42× (γ=0.8, block size S=16). The denoiser was trained on about 2.1T tokens, and the backbone was pretrained on 25T tokens. It has about 60B total parameters and about 3B active parameters per token per tower. It supports three decoding modes: diffusion, simulated AR and AR.

Key Takeaways

  • NVIDIA released Nemotron-Labs-TwoTower,a diffusion language model based on the frozen…
  • It uses a two-tower architecture: the context tower is frozen and the denoiser tower is…
  • Evaluated in BF16 on 2×H100,it retains 98.
  • 7% of the AR baseline quality and improves generation throughput by 2.
  • 8,block size S=16).

Quick Facts

Published
Updated
Source
MarkTechPost(RSS)
Category
Model Releases
AI score
73
Read the original Read the full article at the original source
Why it matters

NVIDIA这个TwoTower把扩散解码接在已有的AR骨干上,几乎无损质量却让吞吐翻倍,并且开源可商用,对批量文本生成的团队是实在的加速工具。

Related entities

Source & copyright

This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.

Original article · AIHOT entry

How to Cite

When citing this article, keep the title, date, and permalink, and credit Orange News as the source.

Orange News: NVIDIA releases Nemotron-Labs-TwoTower open-weight diffusion language model. July 1, 2026. https://new.nat6.net/en/news/cmr1ti4v001xlsl8zovi1yoqa/

View all →