Tutorials & Tips

Hugging Face explains how Accelerate loads and runs very large models with PyTorch

Original title:How 🤗 Accelerate runs very large models thanks to PyTorch

AI summary

Hugging Face published a blog explaining how Accelerate loads and runs very large models that exceed single-machine memory. The core approach uses the PyTorch meta device to create empty models, uses infer_auto_device_map to automatically compute GPU, CPU and disk placement, loads weights shard by shard, and moves weights before and after each forward pass via hooks.

Key Takeaways

  • Hugging Face published a blog explaining how Accelerate loads and runs very large models that…
  • The core approach uses the PyTorch meta device to create empty models,uses infer_auto_device_map…
  • CPU and disk placement,loads weights shard by shard,and moves weights before and after each…

Quick Facts

Published
Updated
Source
Hugging Face 社区博客(混合发现)
Category
Tutorials & Tips
AI score
71
Read the original Read the full article at the original source
Why it matters

原文逐层拆解 Accelerate 借助 PyTorch meta device 与分片加载运行超大模型的机制,并给出可在免费 Colab 复现的做法。

Related entities

Source & copyright

This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.

Original article · AIHOT entry

How to Cite

When citing this article, keep the title, date, and permalink, and credit Orange News as the source.

Orange News: Hugging Face explains how Accelerate loads and runs very large models with PyTorch. September 27, 2022. https://new.nat6.net/en/news/rbxqwwp26sryhiutt1zy63otg/

View all →