Ant Group's inclusionAI open-sources Ling-3.0 series efficient language foundation models
Original title:inclusionAI/Ling-3.0-flash-base-30T
AI summary
Ant Group's inclusionAI open-sourced the Ling-3.0 series language foundation models and released checkpoints from multiple stages including pretraining, mid-training and merging (WSM). The series uses a highly sparse (1/64) MoE architecture, activating only 8 of 512 routed experts per token, with only 5.1B activated parameters (Non-emb) out of 124B total parameters, and natively combines KDA and Gated MLA hybrid linear attention to handle long contexts efficiently.
Key Takeaways
- Ant Group's inclusionAI open-sourced the Ling-3.
- 0 series language foundation models and released checkpoints from multiple stages including…
- The series uses a highly sparse (1/64) MoE architecture,activating only 8 of 512 routed experts…
- 1B activated parameters (Non-emb) out of 124B total parameters,and natively combines KDA and Gated…
Quick Facts
- Published
- Updated
- Category
- Model Releases
- AI score
- 68
把学习率衰减替换为权重合并并发布训练各阶段检查点,使持续预训练与不同衰减策略的离线比较不必逐次重跑,降低实验成本。
Source & copyright
This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.
How to Cite
When citing this article, keep the title, date, and permalink, and credit Orange News as the source.
Orange News: Ant Group's inclusionAI open-sources Ling-3.0 series efficient language foundation models. August 11, 2026. https://new.nat6.net/en/news/cmt0aa8b416yzrodp2cnlw3wj/