7 Commits

Author	SHA1	Message	Date
Laurent Mazare	9fea56d28e	Only optimize float tensors. (#1069)	2023-10-10 09:05:41 +01:00
Laurent Mazare	7529531056	Add the optimizer trait. (#702)	2023-09-01 12:55:39 +01:00
Laurent Mazare	d210c71d77	Set the learning rate. (#687)	2023-08-31 08:03:40 +01:00
Laurent Mazare	0902846f25	Add the AdamW optimizer. (#307) ... * Add the AdamW optimizer. * Add some AdamW test validated against PyTorch.	2023-08-02 14:03:49 +01:00
Laurent Mazare	ff876c2103	Llama more training (#297) ... * Rework the var-builder to handle initializations. * Add some helper functions for layer creation. * Improve the layer initializations. * Get initialized variables. * Precompute the rot embeddings when training lamas.	2023-08-01 19:53:41 +01:00
Laurent Mazare	23e105cd94	Add the gradient for reduce-sum. (#162) ... * Add the gradient for reduce-sum. * And add the gradient for the broadcast ops. * Add some backprop tests. * Add some linear regression example.	2023-07-13 20:14:10 +01:00
Laurent Mazare	ded93a1169	Add the SGD optimizer (#160) ... * Add the nn::optim and some conversion traits. * Add the backward_step function for SGD. * Get the SGD optimizer to work and add a test. * Make the test slighly simpler.	2023-07-13 19:05:44 +01:00