Other

NVIDIA launches DreamGen engine: letting robots "dream" and learn in video generation models

Original title:World modeling for robotics is incredibly hard because (1) control of humanoid robots & 5-finger han...

AI summary

NVIDIA released the DreamGen engine (GR00T Dreams), using video generation models such as Sora/Veo as a neural physics engine. Through a four-step process—fine-tuning the model, simulating parallel worlds, recovering pseudo-actions, and training a foundation model—it generates large-scale synthetic training data for robots. A humanoid robot can learn 22 new behaviors such as pouring and folding from a single pick-and-place task, achieving zero-shot generalization to new verbs and unfamiliar environments (success rates of 43% and 28%, respectively). Compared with traditional graphics engines, this method handles complex interactions such as deformable objects and fluids at constant computational cost. The team plans to fully open-source it within weeks.

Key Takeaways

  • NVIDIA released the DreamGen engine (GR00T Dreams),using video generation models such as Sora/Veo…
  • Through a four-step process—fine-tuning the model,simulating parallel worlds,recovering…
  • A humanoid robot can learn 22 new behaviors such as pouring and folding from a single…
  • Compared with traditional graphics engines,this method handles complex interactions such as…
  • The team plans to fully open-source it within weeks.

Quick Facts

Published
Updated
Source
X:Jim Fan (@DrJimFan)
Category
Other
Read the original Read the full article at the original source
Why it matters

NVIDIA提出用视频生成模型为机器人“造梦”合成训练数据,实现零样本技能泛化

Related entities

Source & copyright

This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.

Original article · AIHOT entry

How to Cite

When citing this article, keep the title, date, and permalink, and credit Orange News as the source.

Orange News: NVIDIA launches DreamGen engine: letting robots "dream" and learn in video generation models. August 5, 2025. https://new.nat6.net/en/news/cmnw1ysuy00zxslc3mc79ibuj/

View all →