Xiaohongshu's RedKnot inference engine splits KV Cache by attention head to accelerate long text
Original title:让 KV Cache「按头分家」:小红书 RedKnot 如何重做长文本推理新引擎
AI summary
RedKnot splits KV Cache along the attention-head dimension, unifying algorithm and storage granularity through three mechanisms: head-class sparsity (local heads account for 83.4%–96.8%), sparse FFN and SegPagedAttention. On 8 H800 GPUs, TTFT is accelerated by up to 1.6–3.54×, single-GPU concurrency improves 4.7–7.8×, and prefill FLOPs are cut by 67%–79.5%. On DeepSeek-V4-Flash, 128K-context TTFT acceleration reaches 5.16×, and KV transfer is reduced by up to 6.3×. Accuracy is typically no lower than 95% of dense F1.
Key Takeaways
- RedKnot splits KV Cache along the attention-head dimension,unifying algorithm and storage…
- 8%),sparse FFN and SegPagedAttention.
- On 8 H800 GPUs,TTFT is accelerated by up to 1.
- 54×,single-GPU concurrency improves 4.
- 8×,and prefill FLOPs are cut by 67%–79.
Quick Facts
- Published
- Updated
- Source
- 公众号:小红书技术(dots.llm)
- Category
- Products & Apps
- AI score
- 72
小红书把 KV Cache 从 token 级拆成按头分家,这个思路让长文本推理的 TTFT 和并发都有数量级提升,开源出来对做推理引擎的同学是个福音。
Related entities
Source & copyright
This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.
How to Cite
When citing this article, keep the title, date, and permalink, and credit Orange News as the source.
Orange News: Xiaohongshu's RedKnot inference engine splits KV Cache by attention head to accelerate long text. June 29, 2026. https://new.nat6.net/en/news/cmqz49v7400e4sldyr126zupc/