mirror of https://github.com/huggingface/candle.git synced 2025-06-15 18:28:24 +00:00

Files

Kyle Birnbaum ba473290da Added DeepseekR1 Qwen7B variant to quantized-qwen2-instruct example (#2843 )

* quantized deepseek qwen generating tokens

* removed is_deepseek from Args and replaced prompt if statement with pattern matching

2025-03-30 10:54:22 +02:00

main.rs

2025-03-30 10:54:22 +02:00

README.md

2024-07-12 10:00:03 +02:00

candle-quantized-qwen2-instruct

Qwen2 is an upgraded version of Qwen1.5, released by Alibaba Cloud.

Running the example

cargo run --example quantized-qwen2-instruct --release -- --prompt "Write a function to count prime numbers up to N."

0.5b, 1.5b, 7b and 72b models are available via --model argument.