Xiaohongshu and Peking University open-source UltraEP: an "optimal" load-balancing solution for large-scale MoE training and inference
AI summary
Xiaohongshu and Peking University open-sourced UltraEP, the first system to bring real-time load balancing based on precise routing information into production, dynamically replicating hot experts in every microbatch and every layer. It reaches 94.3% of ideal performance on average, a 1.49x improvement over state-of-the-art training and inference frameworks. The solution has been deployed in full pretraining of a 288B-parameter MoE model, sustaining no less than 92% of ideal performance.
Key Takeaways
- Xiaohongshu and Peking University open-sourced UltraEP,the first system to bring real-time load…
- It reaches 94.
- 3% of ideal performance on average,a 1.
- 49x improvement over state-of-the-art training and inference frameworks.
- The solution has been deployed in full pretraining of a 288B-parameter MoE model,sustaining no…
Quick Facts
- Published
- Updated
- Source
- 公众号:小红书技术(dots.llm)
- Category
- Products & Apps
- AI score
- 76
动态复制热点专家替代了基于历史窗口的重排,将MoE训推吞吐从理想的一半拉至90%以上,对自研大规模模型的团队而言,是一套可落地的负载均衡方案。
Source & copyright
This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.
How to Cite
When citing this article, keep the title, date, and permalink, and credit Orange News as the source.
Orange News: Xiaohongshu and Peking University open-source UltraEP: an "optimal" load-balancing solution for large-scale MoE training and inference. July 20, 2026. https://new.nat6.net/en/news/cmsdu108a00uerofzae7t96q3/