Papers & Research

How Ling-3.0-flash cuts batch-1 decoding latency by 54% on 4 Blackwell GPUs

Original title:Blog Chasing the Batch-1 Floor: Ling-3.0-flash Speculative Decode on Blackwell Batch-1 decode keeps getting more important. Xiaomi MiMo, for example, announced MiMo-V2.5-Pro UltraSpeed in June, claiming 1,000 tok/s decode on a one-trillion-parameter MoE model. Batch 1 gives an ... RadixArk SGLang Team, Ant Ling Infra Team

AI summary

Ant Ling Infra and the RadixArk SGLang team raised single-request decoding speed for the Ling-3.0-flash hybrid linear attention MoE model from 288 tok/s to 606 tok/s, reducing average TPOT from 3.33 ms to 1.53 ms.

Key Takeaways

  • Ant Ling Infra and the RadixArk SGLang team raised single-request decoding speed for the Ling-3.
  • 0-flash hybrid linear attention MoE model from 288 tok/s to 606 tok/s,reducing average TPOT from 3.
  • 33 ms to 1.

Quick Facts

Published
Updated
Source
LMSYS Blog
Category
Papers & Research
AI score
63
Read the original Read the full article at the original source
Why it matters

对做单请求低延迟推理的团队,可迁移的是先把主机从 GPU 进度上解绑再优化 GPU 关键路径,否则主机的同步等待会变成每步的 GPU 气泡,这与具体模型无关。

Source & copyright

This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.

Original article · AIHOT entry

How to Cite

When citing this article, keep the title, date, and permalink, and credit Orange News as the source.

Orange News: How Ling-3.0-flash cuts batch-1 decoding latency by 54% on 4 Blackwell GPUs. August 21, 2026. https://new.nat6.net/en/news/cmt393qov0kfhro6tuwhxhubl/

View all →