huggingface/candle
1
0
Fork 0
mirror of https://github.com/huggingface/candle.git synced 2025-06-16 02:38:10 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
17313a4226a6c6bde444d28b4be4f0f96d155be7
candle/candle-examples/examples/quantized-qwen3/README.md
MaCAT 9ce4fe6194 Fix docs quantized qwen3 (#2955) 
* fixed docs quantized-qwen3 README

* fixed docs quantized-qwen2-instruct README
2025-05-15 07:58:03 +02:00

549 B
Raw Blame History

candle-quantized-qwen3

Qwen3 is an upgraded version of Qwen2.5, released by Alibaba Cloud.

Running the example

cargo run --example quantized-qwen3 --release -- --prompt "Write a function to count prime numbers up to N."

0.6b is used by default, 1.7b, 4b, 8b, 14b, and 32b models are available via --which argument.

cargo run --example quantized-qwen3 --release -- --which 4b   --prompt "A train is travelling at 120mph, how far does it travel in 3 minutes 30 seconds?"