mirror of
https://github.com/huggingface/candle.git
synced 2025-06-17 02:58:50 +00:00
@ -67,7 +67,7 @@ And then browse to
|
|||||||
- Distributed computing using NCCL.
|
- Distributed computing using NCCL.
|
||||||
- Models out of the box: Llama, Whisper, Falcon, StarCoder...
|
- Models out of the box: Llama, Whisper, Falcon, StarCoder...
|
||||||
- Embed user-defined ops/kernels, such as [flash-attention
|
- Embed user-defined ops/kernels, such as [flash-attention
|
||||||
v2](https://github.com/LaurentMazare/candle/blob/89ba005962495f2bfbda286e185e9c3c7f5300a3/candle-flash-attn/src/lib.rs#L152).
|
v2](https://github.com/huggingface/candle/blob/89ba005962495f2bfbda286e185e9c3c7f5300a3/candle-flash-attn/src/lib.rs#L152).
|
||||||
|
|
||||||
<!--- ANCHOR_END: features --->
|
<!--- ANCHOR_END: features --->
|
||||||
|
|
||||||
|
@ -128,17 +128,17 @@ fn main() -> Result<()> {
|
|||||||
```
|
```
|
||||||
|
|
||||||
Now it works, it is a great way to create your own layers.
|
Now it works, it is a great way to create your own layers.
|
||||||
But most of the classical layers are already implemented in [candle-nn](https://github.com/LaurentMazare/candle/tree/main/candle-nn).
|
But most of the classical layers are already implemented in [candle-nn](https://github.com/huggingface/candle/tree/main/candle-nn).
|
||||||
|
|
||||||
## Using `candle_nn`.
|
## Using `candle_nn`.
|
||||||
|
|
||||||
For instance [Linear](https://github.com/LaurentMazare/candle/blob/main/candle-nn/src/linear.rs) is already there.
|
For instance [Linear](https://github.com/huggingface/candle/blob/main/candle-nn/src/linear.rs) is already there.
|
||||||
This Linear is coded with PyTorch layout in mind, to reuse better existing models out there, so it uses the transpose of the weights and not the weights directly.
|
This Linear is coded with PyTorch layout in mind, to reuse better existing models out there, so it uses the transpose of the weights and not the weights directly.
|
||||||
|
|
||||||
So instead we can simplify our example:
|
So instead we can simplify our example:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cargo add --git https://github.com/LaurentMazare/candle.git candle-nn
|
cargo add --git https://github.com/huggingface/candle.git candle-nn
|
||||||
```
|
```
|
||||||
|
|
||||||
And rewrite our examples using it
|
And rewrite our examples using it
|
||||||
|
@ -5,13 +5,13 @@ Start by creating a new app:
|
|||||||
```bash
|
```bash
|
||||||
cargo new myapp
|
cargo new myapp
|
||||||
cd myapp
|
cd myapp
|
||||||
cargo add --git https://github.com/LaurentMazare/candle.git candle
|
cargo add --git https://github.com/huggingface/candle.git candle-core
|
||||||
```
|
```
|
||||||
|
|
||||||
At this point, candle will be built **without** CUDA support.
|
At this point, candle will be built **without** CUDA support.
|
||||||
To get CUDA support use the `cuda` feature
|
To get CUDA support use the `cuda` feature
|
||||||
```bash
|
```bash
|
||||||
cargo add --git https://github.com/LaurentMazare/candle.git candle --features cuda
|
cargo add --git https://github.com/huggingface/candle.git candle-core --features cuda
|
||||||
```
|
```
|
||||||
|
|
||||||
You can check everything works properly:
|
You can check everything works properly:
|
||||||
|
Reference in New Issue
Block a user