Fine-tuning NVIDIA Cosmos Predict 2.5 with LoRA/DoRA for robot video generation
Original title:Fine-Tuning NVIDIA Cosmos Predict 2.5 with LoRA/DoRA for Robot Video Generation
AI summary
NVIDIA Cosmos Predict 2.5 is a 2B-parameter world model that generates physically plausible video from text, images or video clips. By injecting trainable adapters via LoRA or DoRA into the DiT attention layers (to_q, to_k, to_v, to_out.0) and feed-forward layers while freezing all base weights, parameter-efficient fine-tuning can be completed on a single 80GB GPU, avoiding the high cost and catastrophic forgetting of full fine-tuning. The workflow uses the diffusers and accelerate libraries, fine-tuning on a training set of 92 robot manipulation videos and 50 (prompt, image) test pairs, and shows how to use the fine-tuned model to generate synthetic robot trajectories to support downstream robot learning tasks. It supports single-GPU and multi-GPU training, and switching between domain adapters requires no retraining.
Key Takeaways
- NVIDIA Cosmos Predict 2.
- 5 is a 2B-parameter world model that generates physically plausible video from text,images or…
- By injecting trainable adapters via LoRA or DoRA into the DiT attention layers…
- 0) and feed-forward layers while freezing all base weights,parameter-efficient fine-tuning can be…
- The workflow uses the diffusers and accelerate libraries,fine-tuning on a training set of 92 robot…
Quick Facts
- Published
- Updated
- Source
- Hugging Face 社区博客(混合发现)
- Category
- Tutorials & Tips
- AI score
- 67
这篇教程把微调Cosmos Predict 2.5的方法从头到尾讲清楚了,做机器人合成数据的同行可以直接抄作业,LoRA/DoRA切换也很方便,值得收藏。
Related entities
Source & copyright
This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.
How to Cite
When citing this article, keep the title, date, and permalink, and credit Orange News as the source.
Orange News: Fine-tuning NVIDIA Cosmos Predict 2.5 with LoRA/DoRA for robot video generation. May 19, 2026. https://new.nat6.net/en/news/cmpbfzxd4172bslnzzdo3g1lc/