huggingface/candle
1
0
Fork 0
mirror of https://github.com/huggingface/candle.git synced 2025-06-15 18:28:24 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
main
candle/candle-examples/examples/quantized-qwen3
History
MaCAT 9ce4fe6194 Fix docs quantized qwen3 (#2955) 
* fixed docs quantized-qwen3 README

* fixed docs quantized-qwen2-instruct README
2025-05-15 07:58:03 +02:00
..
main.rs
Qwen3 quantized implementation (#2939)
2025-05-08 15:06:10 +02:00
README.md
Fix docs quantized qwen3 (#2955)
2025-05-15 07:58:03 +02:00

README.md

candle-quantized-qwen3

Qwen3 is an upgraded version of Qwen2.5, released by Alibaba Cloud.

Running the example

cargo run --example quantized-qwen3 --release -- --prompt "Write a function to count prime numbers up to N."

0.6b is used by default, 1.7b, 4b, 8b, 14b, and 32b models are available via --which argument.

cargo run --example quantized-qwen3 --release -- --which 4b   --prompt "A train is travelling at 120mph, how far does it travel in 3 minutes 30 seconds?"