Ant Group's inclusionAI open-sources Ling-3.0 series efficient language base models
Original title:inclusionAI/Ling-3.0-flash-base-midtrain
AI summary
Ant Group's inclusionAI open-sourced the Ling-3.0 series language base models, using a highly sparse (1/64) MoE architecture that activates only 8 of 512 routed experts per token, with 124B total parameters and only 5.1B activated parameters. The series natively integrates linear attention and replaces traditional learning rate decay with weighted checkpoint merging. This release includes checkpoints from multiple training stages including pretraining, mid-training and merging (WSM), supports continued pretraining and fine-tuning, and is under the MIT license.
Key Takeaways
- Ant Group's inclusionAI open-sourced the Ling-3.
- 0 series language base models,using a highly sparse (1/64) MoE architecture that activates only 8…
- 1B activated parameters.
- The series natively integrates linear attention and replaces traditional learning rate decay with…
- This release includes checkpoints from multiple training stages including pretraining,mid-training…
Quick Facts
- Published
- Updated
- Category
- Model Releases
- AI score
- 64
把训练中间态作为可下载 checkpoint 开放,配合 WSM 合并替代学习率衰减,做继续预训练或 MoE 研究时可从中间阶段开始,省去重复前置训练成本。
Source & copyright
This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.
How to Cite
When citing this article, keep the title, date, and permalink, and credit Orange News as the source.
Orange News: Ant Group's inclusionAI open-sources Ling-3.0 series efficient language base models. August 11, 2026. https://new.nat6.net/en/news/cmt0aa8b416z0rodptcwaftq0/