Xiaohongshu and Peking University open-source UltraEP: real-time load balancing for large-scale MoE training and inference
Original title:小红书、北大开源 UltraEP:面向大规模 MoE 训推的「最优」负载均衡方案
AI summary
Xiaohongshu and Peking University proposed UltraEP, the first to bring real-time load balancing based on precise routing information into a production system, dynamically replicating hot experts in every microbatch and every layer. On models such as Qwen3-235B, training throughput averages 94.6% of ideal performance, a 42% improvement over Megatron-LM; inference prefill throughput is 1.56x higher than SGLang.
Key Takeaways
- Xiaohongshu and Peking University proposed UltraEP,the first to bring real-time load balancing…
- On models such as Qwen3-235B,training throughput averages 94.
- 6% of ideal performance,a 42% improvement over Megatron-LM
- inference prefill throughput is 1.
- 56x higher than SGLang.
Quick Facts
- Published
- Updated
- Source
- 公众号:小红书技术(dots.llm)
- Category
- Products & Apps
- AI score
- 68
MoE 训练推理的负载不均是个老痛点,UltraEP 用实时复制热专家的方式把 GPU 利用率拉到 94% 的理想线,代码和论文都开源了,做大规模分布式训练的团队可以直接参考。
Source & copyright
This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.
How to Cite
When citing this article, keep the title, date, and permalink, and credit Orange News as the source.
Orange News: Xiaohongshu and Peking University open-source UltraEP: real-time load balancing for large-scale MoE training and inference. July 20, 2026. https://new.nat6.net/en/news/cmrsy2z9e03wpbitlqgh25fol/