mirror of
https://github.com/huggingface/candle.git
synced 2025-06-16 10:38:54 +00:00
45d5322d62
* Add the Gemma models. * Add the gemma example. * Adapt the RmsNorm. * Get the 2b model to work. * 7b support. * Use the config head dim. * Yet another fix. * Make the matrixes contiguous. * Also get the 7b model to work. * And add to the readme.
28 lines
859 B
Markdown
28 lines
859 B
Markdown
# candle-mistral: 2b and 7b LLMs from Google DeepMind
|
|
|
|
[Gemma](https://ai.google.dev/gemma/docs) is a collection of lightweight open
|
|
models published by Google Deepmind with a 2b and a 7b variant.
|
|
|
|
In order to use the example below, you have to accept the license on the
|
|
[HuggingFace Hub Gemma repo](https://huggingface.co/google/gemma-7b) and set up
|
|
your access token via the [HuggingFace cli login
|
|
command](https://huggingface.co/docs/huggingface_hub/guides/cli#huggingface-cli-login).
|
|
|
|
## Running the example
|
|
|
|
```bash
|
|
$ cargo run --example gemma --release -- --prompt "fn count_primes(max_n: usize)"
|
|
fn count_primes(max_n: usize) -> usize {
|
|
let mut primes = vec![true; max_n];
|
|
for i in 2..=max_n {
|
|
if primes[i] {
|
|
for j in i * i..max_n {
|
|
primes[j] = false;
|
|
}
|
|
}
|
|
}
|
|
primes.len()
|
|
}
|
|
```
|
|
|