Products & Apps

Xiaohongshu and Peking University open-source UltraEP: an "optimal" load-balancing solution for large-scale MoE training and inference

AI summary

Xiaohongshu and Peking University open-sourced UltraEP, the first system to bring real-time load balancing based on precise routing information into production, dynamically replicating hot experts in every microbatch and every layer. It reaches 94.3% of ideal performance on average, a 1.49x improvement over state-of-the-art training and inference frameworks. The solution has been deployed in full pretraining of a 288B-parameter MoE model, sustaining no less than 92% of ideal performance.

Key Takeaways

  • Xiaohongshu and Peking University open-sourced UltraEP,the first system to bring real-time load…
  • It reaches 94.
  • 3% of ideal performance on average,a 1.
  • 49x improvement over state-of-the-art training and inference frameworks.
  • The solution has been deployed in full pretraining of a 288B-parameter MoE model,sustaining no…

Quick Facts

Published
Updated
Source
公众号:小红书技术(dots.llm)
Category
Products & Apps
AI score
76
Read the original Read the full article at the original source
Why it matters

动态复制热点专家替代了基于历史窗口的重排,将MoE训推吞吐从理想的一半拉至90%以上,对自研大规模模型的团队而言,是一套可落地的负载均衡方案。

Source & copyright

This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.

Original article · AIHOT entry

How to Cite

When citing this article, keep the title, date, and permalink, and credit Orange News as the source.

Orange News: Xiaohongshu and Peking University open-source UltraEP: an "optimal" load-balancing solution for large-scale MoE training and inference. July 20, 2026. https://new.nat6.net/en/news/cmsdu108a00uerofzae7t96q3/

View all →