Products & Apps

Xiaohongshu and Peking University open-source UltraEP: real-time load balancing for large-scale MoE training and inference

Original title:小红书、北大开源 UltraEP:面向大规模 MoE 训推的「最优」负载均衡方案

AI summary

Xiaohongshu and Peking University proposed UltraEP, the first to bring real-time load balancing based on precise routing information into a production system, dynamically replicating hot experts in every microbatch and every layer. On models such as Qwen3-235B, training throughput averages 94.6% of ideal performance, a 42% improvement over Megatron-LM; inference prefill throughput is 1.56x higher than SGLang.

Key Takeaways

  • Xiaohongshu and Peking University proposed UltraEP,the first to bring real-time load balancing…
  • On models such as Qwen3-235B,training throughput averages 94.
  • 6% of ideal performance,a 42% improvement over Megatron-LM
  • inference prefill throughput is 1.
  • 56x higher than SGLang.

Quick Facts

Published
Updated
Source
公众号:小红书技术(dots.llm)
Category
Products & Apps
AI score
68
Read the original Read the full article at the original source
Why it matters

MoE 训练推理的负载不均是个老痛点,UltraEP 用实时复制热专家的方式把 GPU 利用率拉到 94% 的理想线,代码和论文都开源了,做大规模分布式训练的团队可以直接参考。

Source & copyright

This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.

Original article · AIHOT entry

How to Cite

When citing this article, keep the title, date, and permalink, and credit Orange News as the source.

Orange News: Xiaohongshu and Peking University open-source UltraEP: real-time load balancing for large-scale MoE training and inference. July 20, 2026. https://new.nat6.net/en/news/cmrsy2z9e03wpbitlqgh25fol/

View all →