huggingface/candle
1
0
Fork 0
mirror of https://github.com/huggingface/candle.git synced 2025-06-15 10:26:33 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
fd7f7242a1e5ebb21d5f17b03a3fa81519818919
candle/candle-examples/examples/mamba/README.md
Laurent Mazare 1e26d539d9 Improved mamba model optimized for inference (#1694) 
* Sketch the mamba model for inference.

* Complete the forward pass.

* Add the mamba example.

* Optimize the selective-scan part.

* Fix a couple shape mismatches and get inference to work.

* Tweak the readmes.

* More readme tweaks.
2024-02-11 17:04:57 +01:00

671 B
Raw Blame History

candle-mamba: Mamba implementation

Candle implementation of Mamba [1] inference only. Mamba is an alternative to the transformer architecture. It leverages State Space Models (SSMs) with the goal of being computationally efficient on long sequences. The implementation is based on mamba.rs.

Compared to the mamba-minimal example, this version is far more efficient but would only work for inference.

Running the example

$ cargo run --example mamba-minimal --release -- --prompt "Mamba is the"