Ant Group's inclusionAI open-sources Ling-3.0 series efficient language foundation models
Original title:inclusionAI/Ling-3.0-tiny-base-30T
AI summary
Ant Group's inclusionAI open-sourced the Ling-3.0 series efficient language foundation models and released checkpoints from pretraining, mid-training and merging (WSM) stages, supporting continued pretraining, fine-tuning and research. Ling-3.0-tiny-base uses a sparse MoE architecture with 128 routed experts, activating only 1.3B parameters per token out of 7.9B total parameters, and natively combines linear attention to handle long contexts efficiently.
Key Takeaways
- Ant Group's inclusionAI open-sourced the Ling-3.
- 0 series efficient language foundation models and released checkpoints from…
- 0-tiny-base uses a sparse MoE architecture with 128 routed experts,activating only 1.
- 3B parameters per token out of 7.
- 9B total parameters,and natively combines linear attention to handle long contexts efficiently.
Quick Facts
- Published
- Updated
- Category
- Model Releases
- AI score
- 63
它把学习率衰减替换为加权检查点合并,基座模型更适合持续预训练和数据扩展,做继续预训练或领域微调时可复用这一训练策略做离线探索。
Source & copyright
This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.
How to Cite
When citing this article, keep the title, date, and permalink, and credit Orange News as the source.
Orange News: Ant Group's inclusionAI open-sources Ling-3.0 series efficient language foundation models. August 11, 2026. https://new.nat6.net/en/news/cmt0aa8b416yxrodpvq5x791t/