NVIDIA launches DreamGen engine: letting robots "dream" and learn in video generation models
Original title:World modeling for robotics is incredibly hard because (1) control of humanoid robots & 5-finger han...
AI summary
NVIDIA released the DreamGen engine (GR00T Dreams), using video generation models such as Sora/Veo as a neural physics engine. Through a four-step process—fine-tuning the model, simulating parallel worlds, recovering pseudo-actions, and training a foundation model—it generates large-scale synthetic training data for robots. A humanoid robot can learn 22 new behaviors such as pouring and folding from a single pick-and-place task, achieving zero-shot generalization to new verbs and unfamiliar environments (success rates of 43% and 28%, respectively). Compared with traditional graphics engines, this method handles complex interactions such as deformable objects and fluids at constant computational cost. The team plans to fully open-source it within weeks.
Key Takeaways
- NVIDIA released the DreamGen engine (GR00T Dreams),using video generation models such as Sora/Veo…
- Through a four-step process—fine-tuning the model,simulating parallel worlds,recovering…
- A humanoid robot can learn 22 new behaviors such as pouring and folding from a single…
- Compared with traditional graphics engines,this method handles complex interactions such as…
- The team plans to fully open-source it within weeks.
Quick Facts
- Published
- Updated
- Source
- X:Jim Fan (@DrJimFan)
- Category
- Other
NVIDIA提出用视频生成模型为机器人“造梦”合成训练数据,实现零样本技能泛化
Related entities
Source & copyright
This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.
How to Cite
When citing this article, keep the title, date, and permalink, and credit Orange News as the source.
Orange News: NVIDIA launches DreamGen engine: letting robots "dream" and learn in video generation models. August 5, 2025. https://new.nat6.net/en/news/cmnw1ysuy00zxslc3mc79ibuj/