DreamGen: letting robots "dream" synthetic training data in video generation models
Original title:What if robots could dream inside a video generative model? Introducing DreamGen, a new engine that ...
AI summary
DreamGen lets robots "dream" synthetic training data in video generation models. By fine-tuning models such as Sora to generate massive neural trajectories (realistic videos plus action labels), robots generalize from a single pick-and-place task to 22 new behaviors such as pouring and folding. In tests at NVIDIA headquarters cafeteria, a humanoid robot's zero-shot success rate for new verbs rose from 0% to 43%, and reached 28% in new environments. Compared with traditional graphics engines, it can handle complex scenarios such as fluids and deformable objects without manual modeling, and the entire pipeline will be fully open-sourced soon.
Key Takeaways
- DreamGen lets robots "dream" synthetic training data in video generation models.
- By fine-tuning models such as Sora to generate massive neural trajectories (realistic videos plus…
- In tests at NVIDIA headquarters cafeteria,a humanoid robot's zero-shot success rate for new verbs…
- Compared with traditional graphics engines,it can handle complex scenarios such as fluids and…
Quick Facts
- Published
- Updated
- Source
- X:Jim Fan (@DrJimFan)
- Category
- Other
NVIDIA 提出 DreamGen:让机器人在视频生成模型中「做梦」合成训练数据,实现强零样本泛化,将开源
Related entities
Source & copyright
This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.
How to Cite
When citing this article, keep the title, date, and permalink, and credit Orange News as the source.
Orange News: DreamGen: letting robots "dream" synthetic training data in video generation models. May 20, 2025. https://new.nat6.net/en/news/cmnw1ysuy0106slc325419zpu/