Unveiling Jamba: AI21's Groundbreaking Hybrid SSM-Transformer Open-Source Model
TEL AVIV, Israel, March 28, 2024 /PRNewswire/ -- AI21, the leader in AI systems for the enterprise, today unveiled Jamba, the world's first production-grade Mamba-style model – integrating Mamba Structured State Space model (SSM) technology with elements of traditional Transformer architecture.
- Jamba marks a significant advancement in large language model (LLM) development, offering unparalleled efficiency, throughput, and performance.
- Jamba revolutionizes the landscape of LLMs by addressing the limitations of pure SSM models and traditional Transformer architectures.
- Jamba features a hybrid architecture that integrates Transformer, Mamba, and mixture-of-experts (MoE) layers, optimizing memory, throughput, and performance simultaneously.
- "We are excited to introduce Jamba, a groundbreaking hybrid architecture that combines the best of Mamba and Transformer technologies," said Or Dagan, VP of Product, at AI21.