MNN adapts to SME2, enabling real-time on-device inference for Qwen3-VL-4B
Original title:端侧 AI 提速 80%?如何让 Qwen3-VL 在手机起飞
AI summary
The MNN inference engine has been deeply adapted to the Arm SME2 instruction set, enabling Qwen3-VL-4B-Instruct to achieve real-time multimodal inference on the vivo X300, which supports SME2. Prefill performance improves by 81%, and Decode performance by 13%. MNN uses a design with compile-time built-ins plus runtime automatic detection, with SME2 acceleration enabled by default. The model is a 4B-parameter vision-language model that supports image-text understanding and dialogue. The officially converted and quantized model can be downloaded and deployed directly through MNN, and developers can enable hardware acceleration with a single compile switch.
Key Takeaways
- The MNN inference engine has been deeply adapted to the Arm SME2 instruction set,enabling…
- Prefill performance improves by 81%,and Decode performance by 13%.
- MNN uses a design with compile-time built-ins plus runtime automatic detection,with SME2…
- The model is a 4B-parameter vision-language model that supports image-text understanding and…
- The officially converted and quantized model can be downloaded and deployed directly through…
Quick Facts
- Published
- Updated
- Source
- 公众号:通义实验室(千问)
- Category
- Tutorials & Tips
- AI score
- 79
这是一份硬核的端侧部署指南,实测数据让 Qwen3-VL 在 SME2 手机上 Prefill 提速超过 80%,做移动端 AI 的团队可以直接抄作业。
Source & copyright
This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.
How to Cite
When citing this article, keep the title, date, and permalink, and credit Orange News as the source.
Orange News: MNN adapts to SME2, enabling real-time on-device inference for Qwen3-VL-4B. June 11, 2026. https://new.nat6.net/en/news/cmqc6ibv400ktslt4e8zhjo27/