Model Releases

Next-generation speculative decoding: DFlash and Spec V2

Original title:Blog The next generation of speculative decoding: DFlash and Spec V2 Using Modal and Z Lab's DFlash speculative decoding models with SGLang’s newly default Spec V2 engine, you can achieve state-of-the-art latencies for LLM inference serving. Our new, jointly-released D... Z Lab, Modal, and SGLang Teams

AI summary

Z Lab, Modal and the SGLang team jointly released the DFlash speculative decoding model and SGLang's default Spec V2 engine. DFlash uses block diffusion plus KV injection to generate entire blocks of draft tokens in parallel; on Qwen 3.5 397B-A17B (BF16) with the HumanEval dataset at concurrency 1, throughput reaches 4.3 times the baseline

Key Takeaways

  • Z Lab,Modal and the SGLang team jointly released the DFlash speculative decoding model and…
  • DFlash uses block diffusion plus KV injection to generate entire blocks of draft tokens in parallel
  • on Qwen 3.
  • 5 397B-A17B (BF16) with the HumanEval dataset at concurrency 1,throughput reaches 4.
  • 3 times the baseline

Quick Facts

Published
Updated
Source
LMSYS Blog
Category
Model Releases
AI score
67
Read the original Read the full article at the original source
Why it matters

DFlash 用并行起草和 KV 注入实现了实测 4.3 倍吞吐,再加上 SGLang Spec V2 引擎优化,推理加速不再是纸上谈兵。做 LLM 部署和推理服务的人,可以直接用这个组合试试。

Related entities

Source & copyright

This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.

Original article · AIHOT entry

How to Cite

When citing this article, keep the title, date, and permalink, and credit Orange News as the source.

Orange News: Next-generation speculative decoding: DFlash and Spec V2. June 15, 2026. https://new.nat6.net/en/news/cmqfhhoq001hksl2aw53w2wz8/

View all →