Add the Mistral 7b model (#983)

* Start sketching the mistral 7b model.

* Add the kv cache.

* Add the decoder layer.

* Add the mistral model.

* Rotary embeddings.

* Add the attention mask.
This commit is contained in:
Laurent Mazare
2023-09-28 14:29:41 +01:00
committed by GitHub
parent 25657804ef
commit c05a348e36
2 changed files with 365 additions and 0 deletions

View File

@ -4,6 +4,7 @@ pub mod dinov2;
pub mod efficientnet;
pub mod falcon;
pub mod llama;
pub mod mistral;
pub mod mixformer;
pub mod quantized_llama;
pub mod quantized_mixformer;