141df4ad2b
Main diffusion loop for the SD example. ( #332 )
2023-08-06 21:39:53 +01:00
166bfd5847
Add the recip op + use it in stable-diffusion. ( #331 )
...
* Add the recip unary op.
* Fix the cuda kernel.
* Use the recip op in sigmoid.
2023-08-06 21:14:52 +01:00
1c062bf06b
Add the ddim scheduler. ( #330 )
2023-08-06 20:44:00 +01:00
d34039e352
Add a stable diffusion example ( #328 )
...
* Start adding a stable-diffusion example.
* Proper computation of the causal mask.
* Add the chunk operation.
* Work in progress: port the attention module.
* Add some dummy modules for conv2d and group-norm, get the attention module to compile.
* Re-enable the 2d convolution.
* Add the embeddings module.
* Add the resnet module.
* Add the unet blocks.
* Add the unet.
* And add the variational auto-encoder.
* Use the pad function from utils.
2023-08-06 17:49:43 +01:00
93cfe5642f
Pyo3 dtype ( #327 )
...
* Better handling of dtypes in pyo3.
* More pyo3 dtype.
2023-08-06 10:17:43 +01:00
88bd3b604a
Add some tensor creation functions to the pyo3 bindings. ( #326 )
2023-08-06 06:50:33 +01:00
b278834267
Support the Accelerate BLAS on macOS. ( #325 )
...
* Add the accelerate feature.
* Ffi tweaks.
2023-08-05 17:25:24 +01:00
0b175fcbbd
Fix the pyo3 build for macos. ( #324 )
...
* Fix the pyo3 build for macos.
* rustfmt fix.
2023-08-05 14:53:57 +01:00
620f83cf66
Add the candle-datasets crate ( #322 )
...
* Move the vision datasets to a separate crate.
* Move the batcher bits.
* Update the readme.
* Move the tiny-stories bits.
---------
Co-authored-by: Jane Doe <jane.doe@example.org >
2023-08-05 08:56:50 +01:00
f7b2a0391d
Transpose the weight matrixes for llama2.c. ( #321 )
2023-08-04 13:32:20 +01:00
8b6f5be1cc
Support q5k quantized data. ( #320 )
2023-08-04 09:51:30 +01:00
df6667ba88
Add some tracing to llama. ( #318 )
2023-08-03 13:52:22 +01:00
a79286885c
Support safetensors weights in llama2.c inference. ( #317 )
2023-08-03 11:10:58 +01:00
74845a4dcd
Use the assert! function as it turns out to be const. ( #316 )
2023-08-03 10:03:43 +01:00
aa76b783eb
Q6K dequantization. ( #315 )
2023-08-03 09:31:20 +01:00
25564357f7
Support some ggml quantized types ( #314 )
...
* Add the quantized types for GGML loading.
* Support quantization for Q2K.
* More quantization support.
* Fix some clippy lints.
2023-08-03 09:16:26 +01:00
634700d84a
Use some consts for ggml values. ( #312 )
2023-08-02 22:03:05 +01:00
e635f18eda
Initial support for reading ggml files. ( #311 )
...
* Start adding support for reading ggml files.
* Compute the proper tensor size.
* Print the read tensors.
* Fix file reading.
2023-08-02 21:59:02 +01:00
dba31473d4
Typos and format and CD only when PR lands.
2023-08-02 19:18:43 +02:00
1b2b32e58d
Remove dead page.t
2023-08-02 18:59:36 +02:00
166f4d1101
s/candle/candle_core/g
2023-08-02 18:40:24 +02:00
ae68635af9
Add small error management.
2023-08-02 18:40:24 +02:00
c11e78b334
Odd rebase artifact.
2023-08-02 18:40:24 +02:00
1b705a426f
Remove duplicate.
2023-08-02 18:40:24 +02:00
a70b95f9e7
Marking unwritten chapters as Draft (disables the link).
2023-08-02 18:40:24 +02:00
a44471a305
Adding more details on how to load things.
...
- Loading with memmap
- Loading a sharded tensor
- Moved some snippets to `candle-examples/src/lib.rs` This is because
managing book specific dependencies is a pain https://github.com/rust-lang/mdBook/issues/706
- This causes a non aligned inclusion https://github.com/rust-lang/mdBook/pull/1856 which we have
to ignore fmt to remove.
mdbook might need some more love :)
2023-08-02 18:40:24 +02:00
45642a8530
Fixing examples.
2023-08-02 18:40:24 +02:00
82464166e4
3rd phase.
2023-08-02 18:40:24 +02:00
52414ba5c8
Bugfix for the llama2 wasm example. ( #310 )
...
* Clean-up the llama2.c wasm example.
* Use a proper tokenizer.
* Add a prompt.
* Bugfix for the llama2 wasm example.
2023-08-02 17:32:36 +01:00
186c308d51
Wasm llama2 tweaks ( #309 )
...
* Clean-up the llama2.c wasm example.
* Use a proper tokenizer.
2023-08-02 15:49:43 +01:00
4f17290ce0
Use AdamW in the llama2 training. ( #308 )
2023-08-02 14:14:02 +01:00
0902846f25
Add the AdamW optimizer. ( #307 )
...
* Add the AdamW optimizer.
* Add some AdamW test validated against PyTorch.
2023-08-02 14:03:49 +01:00
e2acbe1e72
Update the wasm example locations in the readme. ( #306 )
2023-08-02 11:36:43 +01:00
4fe8a02f88
Update the repo location. ( #305 )
2023-08-02 11:12:18 +01:00
03a421f714
Add some missing readme files. ( #304 )
2023-08-02 10:57:12 +01:00
d38943aadc
Add version numbers for all the candle crates ( #303 )
...
* Switch to candle-gemm for the time being.
* Add the missing versions.
2023-08-02 10:52:13 +01:00
51e51da896
Rename the candle crate to candle-core ( #301 )
...
* Rename to candle-core.
* More candle-core renaming.
2023-08-02 08:20:22 +01:00
6e33ff62d6
Update cudarc now that it includes the cublas-f16 and nccl changes. ( #300 )
2023-08-02 05:54:28 +01:00
4b3bd79fbd
Remove the embedding ops in favor of index-select. ( #299 )
...
* Remove the embedding ops in favor of index-select.
* Also remove the cuda kernels.
2023-08-02 05:42:11 +01:00
cc76c63202
Use index-select for the embeddings as it supports backprop. ( #298 )
2023-08-01 20:44:43 +01:00
ff876c2103
Llama more training ( #297 )
...
* Rework the var-builder to handle initializations.
* Add some helper functions for layer creation.
* Improve the layer initializations.
* Get initialized variables.
* Precompute the rot embeddings when training lamas.
2023-08-01 19:53:41 +01:00
a27239f3d9
Add training for the llama2.c example ( #296 )
...
* Rework the commands and run inference by default.
* Add the training module and load the training dataset.
* Random dataset iterator.
* Proper valid-loss computation.
* Compute the evaluation loss.
* Add more substance to the training loop.
2023-08-01 17:23:07 +01:00
babee9f011
Merge pull request #259 from LaurentMazare/book_2
...
Book 2 (load/save)
2023-08-01 17:26:57 +02:00
afb5e24a63
Remove map ownership from save.
2023-08-01 17:19:22 +02:00
89d1fd03e5
Adding new surface for savetensors (global load, global save).
2023-08-01 15:00:38 +02:00
310094310b
Modifying safetensors export to get simple load and save.
2023-08-01 15:00:38 +02:00
836ba3e090
Merge pull request #258 from LaurentMazare/start_book
...
Starting the book.
2023-08-01 14:59:34 +02:00
091e781977
Grammarly pass.
2023-08-01 14:26:02 +02:00
5cead227ef
Adressed comments.
2023-08-01 14:26:02 +02:00
ebd0315623
Typo.
2023-08-01 14:26:02 +02:00
