Products & Apps

SGLang integrates DSpark speculative decoding with confidence-driven variable-length verification

Original title:Blog DSpark in SGLang: Speculative Decoding with Confidence-Driven, Variable-Length Verification Speculative decoding trades extra compute for fewer decode steps, and the trade sours as load grows: at batch size B with K speculative tokens the target verifies B K tokens every step, and past a po... SGLang Team

AI summary

The SGLang team has integrated the DSpark speculative decoding algorithm into its open-source inference engine. The algorithm uses a semi-autoregressive block drafter to generate a group of tokens at once, and employs a confidence head with sequential temperature scaling (STS) to dynamically allocate variable verification lengths per request, thereby trimming invalid verification costs under high load. SGLang supports dense models such as Qwen3 and sparse models such as DeepSeek-V4, handling irregular per-request verification lengths through full CUDA graphs. It offers three verification modes: `static` (full length), `compact` (production path), and `cap-accept` (acceptance-cap measurement). It also introduces zero-overhead scheduling, an online scheduler based on offline cost tables, and fused Triton kernels. In tests on H200 with DeepSeek-V4-Flash, DSpark achieved a better throughput-latency tradeoff than MTP and non-speculative baselines across the full concurrency sweep.

Key Takeaways

  • The SGLang team has integrated the DSpark speculative decoding algorithm into its open-source…
  • The algorithm uses a semi-autoregressive block drafter to generate a group of tokens at once,and…
  • SGLang supports dense models such as Qwen3 and sparse models such as DeepSeek-V4,handling…
  • It offers three verification modes: `static` (full length),`compact` (production path),and…
  • It also introduces zero-overhead scheduling,an online scheduler based on offline cost tables,and…

Quick Facts

Published
Updated
Source
LMSYS Blog
Category
Products & Apps
AI score
63
Read the original Read the full article at the original source
Why it matters

DSpark 这版推测解码的工程落地比论文本身更有看头——SGLang 用 ragged CUDA Graph 和动态调度把 trimm 掉的计算按字节级回收,383 tok/s 的单请求速度对部署方是即时可用的性能增益。

Related entities

Source & copyright

This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.

Original article · AIHOT entry

How to Cite

When citing this article, keep the title, date, and permalink, and credit Orange News as the source.

Orange News: SGLang integrates DSpark speculative decoding with confidence-driven variable-length verification. July 6, 2026. https://new.nat6.net/en/news/cmr9h98co0470slsmqqc2ilv1/

View all →