Products & Apps

SGLang introduces Waterfill and LPLB to improve DeepEP MoE load balancing

Original title:Blog Improving DeepEP MoE Load Balance in SGLang with Waterfill and LPLB Mixture-of-Experts (MoE) models rely on Expert Parallelism (EP) to scale inference across multiple GPUs. In SGLang, DeepEP and EPLB provide high-performance serving under EP, but the workload seen by ... NVIDIA Team

AI summary

SGLang added two scheduling-time load-balancing methods for DeepEP MoE inference. Waterfill assigns shared experts to lower-load ranks, raising total throughput by 1.48% to 4.66% under DeepSeek-V3/R1 serving loads, and on DeepSeek V4 lifting the best point from 49,253 tok/s to 51,677 tok/s (+4.92%). LPLB uses linear programming to optimize token routing for redundant expert replicas, and together with EPLB achieves throughput gains of 0.84% to 7.34% on the same cluster.

Key Takeaways

  • SGLang added two scheduling-time load-balancing methods for DeepEP MoE inference.
  • Waterfill assigns shared experts to lower-load ranks,raising total throughput by 1.
  • 66% under DeepSeek-V3/R1 serving loads,and on DeepSeek V4 lifting the best point from 49,253 tok/s…
  • LPLB uses linear programming to optimize token routing for redundant expert replicas,and together…
  • 34% on the same cluster.

Quick Facts

Published
Updated
Source
LMSYS Blog
Category
Products & Apps
AI score
58
Read the original Read the full article at the original source
Why it matters

SGLang 引入 Waterfill 和 LPLB 两种负载均衡算法,实测 DeepSeek V3/R1 和 V4 吞吐提升最高 7%,用 SGLang 跑 MoE 推理的开发者值得一试。

Related entities

Source & copyright

This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.

Original article · AIHOT entry

How to Cite

When citing this article, keep the title, date, and permalink, and credit Orange News as the source.

Orange News: SGLang introduces Waterfill and LPLB to improve DeepEP MoE load balancing. June 26, 2026. https://new.nat6.net/en/news/cmqwtlsg0001nslp0zt3eyzhr/

View all →