huggingface/candle
1
0
Fork 0
mirror of https://github.com/huggingface/candle.git synced 2025-06-20 12:06:35 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,038 Commits 74 Branches 16 Tags
ee8bb1bde1a44738c314dfaacba743f4eabf917c
Commit Graph

4 Commits

Author SHA1 Message Date
Laurent Mazare
0902846f25 Add the AdamW optimizer. (#307) 
* Add the AdamW optimizer.

* Add some AdamW test validated against PyTorch.
 2023-08-02 14:03:49 +01:00
Laurent Mazare
ff876c2103 Llama more training (#297) 
* Rework the var-builder to handle initializations.

* Add some helper functions for layer creation.

* Improve the layer initializations.

* Get initialized variables.

* Precompute the rot embeddings when training lamas.
 2023-08-01 19:53:41 +01:00
Laurent Mazare
23e105cd94 Add the gradient for reduce-sum. (#162) 
* Add the gradient for reduce-sum.

* And add the gradient for the broadcast ops.

* Add some backprop tests.

* Add some linear regression example.
 2023-07-13 20:14:10 +01:00
Laurent Mazare
ded93a1169 Add the SGD optimizer (#160) 
* Add the nn::optim and some conversion traits.

* Add the backward_step function for SGD.

* Get the SGD optimizer to work and add a test.

* Make the test slighly simpler.
 2023-07-13 19:05:44 +01:00