3 Commits

Author	SHA1	Message	Date
Laurent Mazare	4f17290ce0	Use AdamW in the llama2 training. (#308)	2023-08-02 14:14:02 +01:00
Laurent Mazare	ff876c2103	Llama more training (#297) ... * Rework the var-builder to handle initializations. * Add some helper functions for layer creation. * Improve the layer initializations. * Get initialized variables. * Precompute the rot embeddings when training lamas.	2023-08-01 19:53:41 +01:00
Laurent Mazare	a27239f3d9	Add training for the llama2.c example (#296) ... * Rework the commands and run inference by default. * Add the training module and load the training dataset. * Random dataset iterator. * Proper valid-loss computation. * Compute the evaluation loss. * Add more substance to the training loop.	2023-08-01 17:23:07 +01:00