SGLang introduces Waterfill and LPLB to improve DeepEP MoE load balancing
Original title:Blog Improving DeepEP MoE Load Balance in SGLang with Waterfill and LPLB Mixture-of-Experts (MoE) models rely on Expert Parallelism (EP) to scale inference across multiple GPUs. In SGLang, DeepEP and EPLB provide high-performance serving under EP, but the workload seen by ... NVIDIA Team
AI summary
SGLang added two scheduling-time load-balancing methods for DeepEP MoE inference. Waterfill assigns shared experts to lower-load ranks, raising total throughput by 1.48% to 4.66% under DeepSeek-V3/R1 serving loads, and on DeepSeek V4 lifting the best point from 49,253 tok/s to 51,677 tok/s (+4.92%). LPLB uses linear programming to optimize token routing for redundant expert replicas, and together with EPLB achieves throughput gains of 0.84% to 7.34% on the same cluster.
Key Takeaways
- SGLang added two scheduling-time load-balancing methods for DeepEP MoE inference.
- Waterfill assigns shared experts to lower-load ranks,raising total throughput by 1.
- 66% under DeepSeek-V3/R1 serving loads,and on DeepSeek V4 lifting the best point from 49,253 tok/s…
- LPLB uses linear programming to optimize token routing for redundant expert replicas,and together…
- 34% on the same cluster.
Quick Facts
- Published
- Updated
- Source
- LMSYS Blog
- Category
- Products & Apps
- AI score
- 58
SGLang 引入 Waterfill 和 LPLB 两种负载均衡算法,实测 DeepSeek V3/R1 和 V4 吞吐提升最高 7%,用 SGLang 跑 MoE 推理的开发者值得一试。
Related entities
Source & copyright
This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.
How to Cite
When citing this article, keep the title, date, and permalink, and credit Orange News as the source.
Orange News: SGLang introduces Waterfill and LPLB to improve DeepEP MoE load balancing. June 26, 2026. https://new.nat6.net/en/news/cmqwtlsg0001nslp0zt3eyzhr/