Other

DreamGen: letting robots "dream" synthetic training data in video generation models

Original title:What if robots could dream inside a video generative model? Introducing DreamGen, a new engine that ...

AI summary

DreamGen lets robots "dream" synthetic training data in video generation models. By fine-tuning models such as Sora to generate massive neural trajectories (realistic videos plus action labels), robots generalize from a single pick-and-place task to 22 new behaviors such as pouring and folding. In tests at NVIDIA headquarters cafeteria, a humanoid robot's zero-shot success rate for new verbs rose from 0% to 43%, and reached 28% in new environments. Compared with traditional graphics engines, it can handle complex scenarios such as fluids and deformable objects without manual modeling, and the entire pipeline will be fully open-sourced soon.

Key Takeaways

  • DreamGen lets robots "dream" synthetic training data in video generation models.
  • By fine-tuning models such as Sora to generate massive neural trajectories (realistic videos plus…
  • In tests at NVIDIA headquarters cafeteria,a humanoid robot's zero-shot success rate for new verbs…
  • Compared with traditional graphics engines,it can handle complex scenarios such as fluids and…

Quick Facts

Published
Updated
Source
X:Jim Fan (@DrJimFan)
Category
Other
Read the original Read the full article at the original source
Why it matters

NVIDIA 提出 DreamGen:让机器人在视频生成模型中「做梦」合成训练数据,实现强零样本泛化,将开源

Related entities

Source & copyright

This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.

Original article · AIHOT entry

How to Cite

When citing this article, keep the title, date, and permalink, and credit Orange News as the source.

Orange News: DreamGen: letting robots "dream" synthetic training data in video generation models. May 20, 2025. https://new.nat6.net/en/news/cmnw1ysuy0106slc325419zpu/

View all →