What problem does it solve?
Mamba solves the problem of complex and resource-intensive sequence modeling by offering a state-space model with linear complexity, which is significantly faster and more memory-efficient compared to traditional Transformer models.
Core Features & Use Cases
- Linear Complexity: Achieves O(n) complexity, allowing for much faster inference on long sequences.
- Long Context Support: Efficiently handles million-token sequences without the need for large memory footprint.
- Efficient Inference: Up to 5× faster inference compared to Transformers, with no attention overhead.
- Hardware-Aware Design: Optimized for NVIDIA GPUs and CUDA, reducing memory usage and increasing throughput.
- Use Case: Ideal for applications requiring fast, efficient sequence modeling, such as language models, audio processing, and genomics.
Quick Start
To use Mamba for sequence modeling, first install the Mamba library:
pip install mamba-ssm
Then, load and use the Mamba model in your Python code:
import torch
from mamba_ssm import Mamba
batch, length, dim = 2, 64, 16
x = torch.randn(batch, length, dim).to("cuda")
model = Mamba(
d_model=dim, # Model dimension
d_state=16, # SSM state dimension
d_conv=4, # Conv1d kernel size
expand=2 # Expansion factor
).to("cuda")
y = model(x) # O(n) complexity!
assert y.shape == x.shape
