SGLang integrates DSpark speculative decoding with confidence-driven variable-length verification
Original title:Blog DSpark in SGLang: Speculative Decoding with Confidence-Driven, Variable-Length Verification Speculative decoding trades extra compute for fewer decode steps, and the trade sours as load grows: at batch size B with K speculative tokens the target verifies B K tokens every step, and past a po... SGLang Team
AI summary
The SGLang team has integrated the DSpark speculative decoding algorithm into its open-source inference engine. The algorithm uses a semi-autoregressive block drafter to generate a group of tokens at once, and employs a confidence head with sequential temperature scaling (STS) to dynamically allocate variable verification lengths per request, thereby trimming invalid verification costs under high load. SGLang supports dense models such as Qwen3 and sparse models such as DeepSeek-V4, handling irregular per-request verification lengths through full CUDA graphs. It offers three verification modes: `static` (full length), `compact` (production path), and `cap-accept` (acceptance-cap measurement). It also introduces zero-overhead scheduling, an online scheduler based on offline cost tables, and fused Triton kernels. In tests on H200 with DeepSeek-V4-Flash, DSpark achieved a better throughput-latency tradeoff than MTP and non-speculative baselines across the full concurrency sweep.
Key Takeaways
- The SGLang team has integrated the DSpark speculative decoding algorithm into its open-source…
- The algorithm uses a semi-autoregressive block drafter to generate a group of tokens at once,and…
- SGLang supports dense models such as Qwen3 and sparse models such as DeepSeek-V4,handling…
- It offers three verification modes: `static` (full length),`compact` (production path),and…
- It also introduces zero-overhead scheduling,an online scheduler based on offline cost tables,and…
Quick Facts
- Published
- Updated
- Source
- LMSYS Blog
- Category
- Products & Apps
- AI score
- 63
DSpark 这版推测解码的工程落地比论文本身更有看头——SGLang 用 ragged CUDA Graph 和动态调度把 trimm 掉的计算按字节级回收,383 tok/s 的单请求速度对部署方是即时可用的性能增益。
Related entities
Source & copyright
This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.
How to Cite
When citing this article, keep the title, date, and permalink, and credit Orange News as the source.
Orange News: SGLang integrates DSpark speculative decoding with confidence-driven variable-length verification. July 6, 2026. https://new.nat6.net/en/news/cmr9h98co0470slsmqqc2ilv1/