Products & Apps

vLLM Releases vllm-metal v0.28.0: Concurrent Inference Serving on Apple Silicon

Original title:Announcing vllm-metal: Concurrent Serving on Apple Silicon

AI summary

vLLM officially released vllm-metal v0.28.0, bringing vLLM's V1 scheduler, paged KV cache and OpenAI-compatible server to Apple Silicon, with models executed by MLX and Metal; the version number aligns with upstream vLLM.

Key Takeaways

  • vLLM officially released vllm-metal v0.
  • 0,bringing vLLM's V1 scheduler,paged KV cache and OpenAI-compatible server to Apple Silicon,with…
  • the version number aligns with upstream vLLM.

Quick Facts

Published
Updated
Source
vLLM 官方博客(RSS)
Category
Products & Apps
AI score
74
Read the original Read the full article at the original source
Why it matters

官方首次发布 vllm-metal,用 packed varlen attention 和 paged KV 解决 Mac 上并发请求的服务问题,并给出可复现的并发基准数据。

Related entities

Source & copyright

This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.

Original article · AIHOT entry

How to Cite

When citing this article, keep the title, date, and permalink, and credit Orange News as the source.

Orange News: vLLM Releases vllm-metal v0.28.0: Concurrent Inference Serving on Apple Silicon. September 22, 2026. https://new.nat6.net/en/news/cmue9uxe20smtroghgwne9l3u/

View all →