mirror of
https://github.com/huggingface/candle.git
synced 2025-06-16 10:38:54 +00:00
349c3e806a
* Support embedding model gte-Qwen1.5-7B-instruct This is a text embedding model based on Qwen2. They share same model architecture except the last MLP module. This commit brings in minimal modification of the old Qwen2 implementation to support both models. An example is provided, and had been verified according to the official PyTorch implementation. * Avoid doing the 'last-token filtering' based on the absence of attention mask. --------- Co-authored-by: Laurent <laurent.mazare@gmail.com>
candle-qwen: large language model series from Alibaba Cloud
Qwen 1.5 is a series of large language models that provide strong performances on English and Chinese.
- Blog post introducing Qwen1.5.
- Model card on the HuggingFace Hub.
- Blog post for the mixture-of-experts (MoE) variant.
Running the example
$ cargo run --example qwen --release -- --prompt "Hello there "
Various model sizes are available via the --model argument, including the MoE
variant.
$ cargo run --example qwen --release -- --model moe-a2.7b --prompt 'def print_prime(n: int): '
def print_prime(n: int): # n is the number of primes to be printed
for i in range(2, n + 1):
if all(i % j != 0 for j in range(2, i)):
print(i)