Model Releases

Zyphra releases 7B hybrid architecture foundation model Zamba

Original title:ZAMBA

AI summary

Zyphra released the 7B foundation model Zamba, which uses a hybrid architecture of Mamba blocks plus global attention layers with weights shared every 6 layers.

Quick Facts

Published
Updated
Source
Zyphra Research(网页)
Category
Model Releases
AI score
61
Read the original Read the full article at the original source
Why it matters

官方原文给出架构设计、训练规模和开放 checkpoints 细节,读者可以据此评估 Mamba 混合架构在本地部署上的取舍。

Source & copyright

This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.

Original article · AIHOT entry

How to Cite

When citing this article, keep the title, date, and permalink, and credit Orange News as the source.

Orange News: Zyphra releases 7B hybrid architecture foundation model Zamba. April 16, 2024. https://new.nat6.net/en/news/cmtyo1ljj03asrog0h47mq5so/

View all →