huggingface/candle
1
0
Fork 0
mirror of https://github.com/huggingface/candle.git synced 2025-06-20 12:06:35 +00:00
Code Issues Packages Projects Releases Wiki Activity

add quantized qwen2 (#2329)

Browse Source 
* add quantized version of qwen2 and corresponding example for qwen2-instruct

* fix quantized qwen2 clippy error
This commit is contained in:
Zhuo Jinggang
2024-07-12 16:00:03 +08:00
committed by GitHub
parent a226a9736b
commit c63048d374
4 changed files with 641 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
candle-examples/examples/quantized-qwen2-instruct/README.md Normal file
View File

@ -0,0 +1,11 @@
# candle-quantized-qwen2-instruct
[Qwen2]((https://qwenlm.github.io/blog/qwen2/)) is an upgraded version of Qwen1.5, released by Alibaba Cloud.
## Running the example
```bash
cargo run --example quantized-qwen2-instruct --release -- --prompt "Write a function to count prime numbers up to N."
```
0.5b, 1.5b, 7b and 72b models are available via `--model` argument.