Products & Apps

Perplexity open-sources Unigram tokenizer to cut CPU usage

Original title:We're open-sourcing the Unigram tokenizer we rebuilt to reduce CPU utilization by 5-6x. Small reran...

AI summary

We have open-sourced a rebuilt Unigram tokenizer that cuts CPU usage by 5-6x. Small rerankers and embedding models run in single-digit milliseconds on GPU, making CPU tokenization a significant part of total latency. http://github.com/perplexityai/pplx-garden

Key Takeaways

  • We have open-sourced a rebuilt Unigram tokenizer that cuts CPU usage by 5-6x.
  • Small rerankers and embedding models run in single-digit milliseconds on GPU
  • making CPU tokenization a significant part of total latency.

Quick Facts

Published
Updated
Source
X:Perplexity (@perplexity_ai)
Category
Products & Apps
AI score
68
Read the original Read the full article at the original source
Why it matters

Perplexity 开源的这个 tokenizer 能把 CPU 利用率砍掉五六倍,做实时推理的团队值得立马试试,对延迟敏感的场景是实打实的优化。

Related entities

Source & copyright

This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.

Original article · AIHOT entry

How to Cite

When citing this article, keep the title, date, and permalink, and credit Orange News as the source.

Orange News: Perplexity open-sources Unigram tokenizer to cut CPU usage. May 27, 2026. https://new.nat6.net/en/news/cmpo929df03ztslv4z9v93608/

View all →