2 Commits

Author	SHA1	Message	Date
Laurent Mazare	0902846f25	Add the AdamW optimizer. (#307) ... * Add the AdamW optimizer. * Add some AdamW test validated against PyTorch.	2023-08-02 14:03:49 +01:00
Laurent Mazare	3eb2bc6d07	Softmax numerical stability. (#267) ... * Softmax numerical stability. * Fix the flash-attn test.	2023-07-28 13:13:01 +01:00