Google releases Gemma 4 QAT checkpoints for local running on consumer GPUs and mobile devices
Original title:New @GoogleGemma 4 QAT (Quantization-Aware Training) checkpoints are here, so you can run models loc...
AI summary
Google released Gemma 4 quantization-aware training (QAT) checkpoints that support local running on consumer GPUs and mobile devices with minimal quality loss. The new checkpoints are available in GGUF (Q4_0) format, covering all sizes and draft models for optimal local performance. A custom mobile mode uses a mixed-precision approach to compress Gemma 4 to under 1GB, including 2-bit decoder layers, optimized KV cache, and static activations. By simulating compression during training rather than post-training quantization, it significantly reduces memory usage and speeds up decoding while maintaining reasoning quality.
Key Takeaways
- Google released Gemma 4 quantization-aware training (QAT) checkpoints that support local running…
- The new checkpoints are available in GGUF (Q4_0) format,covering all sizes and draft models for…
- A custom mobile mode uses a mixed-precision approach to compress Gemma 4 to under 1GB,including…
- By simulating compression during training rather than post-training quantization,it significantly…
Quick Facts
- Published
- Updated
- Category
- Model Releases
- AI score
- 72
Gemma 4 的量化版把模型压到 1GB 以下,手机本地跑大模型的门槛又低了一大截。Google 这次没用传统的训练后量化,而是把压缩直接嵌进训练里,效果比 PTQ 好一截,搞端侧部署的可以拿 checkpoint 试起来了。
Related entities
Source & copyright
This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.
How to Cite
When citing this article, keep the title, date, and permalink, and credit Orange News as the source.
Orange News: Google releases Gemma 4 QAT checkpoints for local running on consumer GPUs and mobile devices. June 6, 2026. https://new.nat6.net/en/news/cmq166ise0cu5sltrv7a12syq/