Papers & Research

Pushing the limits of DeepSeek-V4-Pro serving: multi-scenario optimization on H20

Original title:Blog Pushing the Limits of Serving DeepSeek-V4-Pro DeepSeek-V4-Pro is a 1.6-trillion-parameter Mixture-of-Experts (MoE) model released with both FP8 and FP4 weights. Models at this scale naturally benefit from accelerators such as NVIDIA Blackwell GPU... Tianyu Zhang, Yusong Gao, Yun Zhang

AI summary

The LMSYS team approached B300 performance on H20 GPUs for the 1.6-trillion-parameter MoE model DeepSeek-V4-Pro through scenario-specific serving configurations. A single-node H20-141GB reference implementation reached 271 output tokens/s, narrowing the performance gap with B300's 383.7 tokens/s to 1.42x.

Key Takeaways

  • The LMSYS team approached B300 performance on H20 GPUs for the 1.
  • 6-trillion-parameter MoE model DeepSeek-V4-Pro through scenario-specific serving configurations.
  • A single-node H20-141GB reference implementation reached 271 output tokens/s,narrowing the…
  • 7 tokens/s to 1.

Quick Facts

Published
Updated
Source
LMSYS Blog
Category
Papers & Research
AI score
73
Read the original Read the full article at the original source
Why it matters

文章提出按上下文长度与并发需求分场景选择 prefill/decode 配置,并用 Humming 压缩和 Online C128 释放 HBM,使 H20 也能服务 1M 上下文,为受限硬件上的大规模推理提供了可复用的工程思路。

Related entities

Source & copyright

This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.

Original article · AIHOT entry

How to Cite

When citing this article, keep the title, date, and permalink, and credit Orange News as the source.

Orange News: Pushing the limits of DeepSeek-V4-Pro serving: multi-scenario optimization on H20. August 19, 2026. https://new.nat6.net/en/news/cmt0e7sxo01eiro2o4l4y9jmh/

View all →