Zyphra releases 7B hybrid architecture foundation model Zamba
Original title:ZAMBA
AI summary
Zyphra released the 7B foundation model Zamba, which uses a hybrid architecture of Mamba blocks plus global attention layers with weights shared every 6 layers.
Quick Facts
- Published
- Updated
- Source
- Zyphra Research(网页)
- Category
- Model Releases
- AI score
- 61
Why it matters
官方原文给出架构设计、训练规模和开放 checkpoints 细节,读者可以据此评估 Mamba 混合架构在本地部署上的取舍。
Source & copyright
This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.
How to Cite
When citing this article, keep the title, date, and permalink, and credit Orange News as the source.
Orange News: Zyphra releases 7B hybrid architecture foundation model Zamba. April 16, 2024. https://new.nat6.net/en/news/cmtyo1ljj03asrog0h47mq5so/