mirror of
https://github.com/huggingface/candle.git
synced 2025-06-16 18:48:51 +00:00
1e26d539d9
* Sketch the mamba model for inference. * Complete the forward pass. * Add the mamba example. * Optimize the selective-scan part. * Fix a couple shape mismatches and get inference to work. * Tweak the readmes. * More readme tweaks.
18 lines
671 B
Markdown
18 lines
671 B
Markdown
# candle-mamba: Mamba implementation
|
|
|
|
Candle implementation of *Mamba* [1] inference only. Mamba is an alternative to
|
|
the transformer architecture. It leverages State Space Models (SSMs) with the
|
|
goal of being computationally efficient on long sequences. The implementation is
|
|
based on [mamba.rs](https://github.com/LaurentMazare/mamba.rs).
|
|
|
|
- [1]. [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://arxiv.org/abs/2312.00752).
|
|
|
|
Compared to the mamba-minimal example, this version is far more efficient but
|
|
would only work for inference.
|
|
## Running the example
|
|
|
|
```bash
|
|
$ cargo run --example mamba-minimal --release -- --prompt "Mamba is the"
|
|
```
|
|
|