huggingface/candle
1
0
Fork 0
mirror of https://github.com/huggingface/candle.git synced 2025-06-16 18:48:51 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
37692065837b2d635d2eb9e7ce3e4e1d439f7028
candle/candle-examples/examples/quantized-qwen2-instruct
History
Zhuo Jinggang c63048d374 add quantized qwen2 (#2329) 
* add quantized version of qwen2 and corresponding example for qwen2-instruct

* fix quantized qwen2 clippy error
2024-07-12 10:00:03 +02:00
..
main.rs
add quantized qwen2 (#2329)
2024-07-12 10:00:03 +02:00
README.md
add quantized qwen2 (#2329)
2024-07-12 10:00:03 +02:00

README.md

candle-quantized-qwen2-instruct

Qwen2 is an upgraded version of Qwen1.5, released by Alibaba Cloud.

Running the example

cargo run --example quantized-qwen2-instruct --release -- --prompt "Write a function to count prime numbers up to N."

0.5b, 1.5b, 7b and 72b models are available via --model argument.