Mamba-3
#62 v Open-source jazykové modelyunknown · v3 · od 2026-03-17 · 2× · naposledy 30. 6. 2026
5
Momentum
Mamba-3 is an open-source state space model (SSM) published on March 16/17, 2026 as a conference paper at ICLR 2026. It introduces three core innovations over Mamba-2: an exponential-trapezoidal discretization for more expressive recurrence, complex-valued state transitions for improved state tracking, and a Multi-Input Multi-Output (MIMO) formulation that increases hardware utilization during decoding without raising decode latency. The model is released in two variants (SISO and MIMO) under the Apache 2.0 license. At 1.5B parameters, Mamba-3 (MIMO) outperforms all Transformer baselines and previous linear sequence models on standard downstream benchmarks.
Vývoj momenta
04.04.03.07.
Vlastnosti
| Inference Speed | Up to 7x faster than Transformer on long sequences; MIMO variant improves hardware utilization during decoding without increasing decode latency compared to Mamba-2. |
| Context Window | 2,048 tokens (training context length used to pretrain all models) |
| Model Size (Parameters) | Tested scales: 360M, 760M, 1B, 1.5B parameters (main benchmark scale: 1.5B). Both variants: SISO and MIMO. |
| Price Tier | Free / Open Source (Apache 2.0); code on GitHub, weights on Hugging Face (state-spaces/mamba) |