Papers & Research

Xiaohongshu, Peking University and SJTU propose HYPIC, bringing position-independent caching to hybrid-attention large models and cutting first-token latency by 3.25x

Original title:首 token 延迟降 3.25 倍!小红书联合北大、上交提出 HYPIC,让混合注意力大模型用上「位置无关缓存」

AI summary

Xiaohongshu, Peking University and Shanghai Jiao Tong University proposed HYPIC, the first serving system to achieve position-independent caching on hybrid-attention large models. Across 4 production-grade hybrid-attention models and 5 workloads, HYPIC reduces first-token latency (TTFT) by 3.25x on average and raises sustainable QPS by 1.66x under the same SLO, while task quality differs from full recomputation by only 1.71 points. The system uses a cache-segment cumulative transfer operator to combine linear-layer states in constant time and a stitching window to fix cross-segment alignment in full-attention layers.

Key Takeaways

  • Xiaohongshu,Peking University and Shanghai Jiao Tong University proposed HYPIC,the first serving…
  • Across 4 production-grade hybrid-attention models and 5 workloads,HYPIC reduces first-token…
  • 25x on average and raises sustainable QPS by 1.
  • 66x under the same SLO,while task quality differs from full recomputation by only 1.
  • 71 points.

Quick Facts

Published
Updated
Source
公众号:小红书技术(dots.llm)
Category
Papers & Research
AI score
75
Read the original Read the full article at the original source
Why it matters

此前位置无关缓存无法用于混合注意力模型,HYPIC 通过缓存转移算子与缝合窗口首次实现,RAG 场景 TTFT 降低 3.25 倍,开源代码为推理系统提供了可复用的加速路径。

Source & copyright

This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.

Original article · AIHOT entry

How to Cite

When citing this article, keep the title, date, and permalink, and credit Orange News as the source.

Orange News: Xiaohongshu, Peking University and SJTU propose HYPIC, bringing position-independent caching to hybrid-attention large models and cutting first-token latency by 3.25x. July 16, 2026. https://new.nat6.net/en/news/cmsdu108a00ufrofz01bxk6m3/

View all →