Add the Mixtral model. (#1437)

* Add the Mixtral model.

* Add more of the mixtral layers.

* Add the final layers for mixtral.

* Sketch the expert selection.

* Add some expert routing logic.

* Hopefully finish the routing logic for mixtral.

* Add the mixtral example.

* Fix the weight filenames.

* Bugfix.

* Another fix.

* Yet another fix + remove the unused pragma.

* Shape fix.

* Add a readme.

candle-transformers/src/models/mod.rs

@@ -14,6 +14,7 @@ pub mod llama2_c_weights;
 pub mod marian;
 pub mod mistral;
 pub mod mixformer;
+pub mod mixtral;
 pub mod mpt;
 pub mod persimmon;
 pub mod quantized_blip;