Products & Apps

Xiaohongshu's RedKnot inference engine splits KV Cache by attention head to accelerate long text

Original title:让 KV Cache「按头分家」:小红书 RedKnot 如何重做长文本推理新引擎

AI summary

RedKnot splits KV Cache along the attention-head dimension, unifying algorithm and storage granularity through three mechanisms: head-class sparsity (local heads account for 83.4%–96.8%), sparse FFN and SegPagedAttention. On 8 H800 GPUs, TTFT is accelerated by up to 1.6–3.54×, single-GPU concurrency improves 4.7–7.8×, and prefill FLOPs are cut by 67%–79.5%. On DeepSeek-V4-Flash, 128K-context TTFT acceleration reaches 5.16×, and KV transfer is reduced by up to 6.3×. Accuracy is typically no lower than 95% of dense F1.

Key Takeaways

  • RedKnot splits KV Cache along the attention-head dimension,unifying algorithm and storage…
  • 8%),sparse FFN and SegPagedAttention.
  • On 8 H800 GPUs,TTFT is accelerated by up to 1.
  • 54×,single-GPU concurrency improves 4.
  • 8×,and prefill FLOPs are cut by 67%–79.

Quick Facts

Published
Updated
Source
公众号:小红书技术(dots.llm)
Category
Products & Apps
AI score
72
Read the original Read the full article at the original source
Why it matters

小红书把 KV Cache 从 token 级拆成按头分家,这个思路让长文本推理的 TTFT 和并发都有数量级提升,开源出来对做推理引擎的同学是个福音。

Related entities

Source & copyright

This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.

Original article · AIHOT entry

How to Cite

When citing this article, keep the title, date, and permalink, and credit Orange News as the source.

Orange News: Xiaohongshu's RedKnot inference engine splits KV Cache by attention head to accelerate long text. June 29, 2026. https://new.nat6.net/en/news/cmqz49v7400e4sldyr126zupc/

View all →