mirror of https://github.com/huggingface/candle.git synced 2025-06-19 11:56:45 +00:00

Files

Laurent Mazare deee7612da Quantized version of mistral. (#1009 )

* Quantized version of mistral.

* Integrate the quantized mistral variant.

* Use the quantized weight files.

* Tweak the quantization command.

* Fix the dtype when computing the rotary embeddings.

* Update the readme with the quantized version.

* Fix the decoding of the remaining tokens.

2023-09-30 18:25:47 +01:00