2 Commits

Author	SHA1	Message	Date
Laurent Mazare	3eb2bc6d07	Softmax numerical stability. (#267) ... * Softmax numerical stability. * Fix the flash-attn test.	2023-07-28 13:13:01 +01:00
Laurent Mazare	4f92420132	Add some flash attn test (#253) ... * Add some flash-attn test. * Add the cpu test. * Fail when the head is not a multiple of 8. * Polish the flash attention test.	2023-07-26 20:56:00 +01:00