huggingface/candle
1
0
Fork 0
mirror of https://github.com/huggingface/candle.git synced 2025-06-16 18:48:51 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,079 Commits 74 Branches 16 Tags
operators-argmin-argmax-leakyrelu
Commit Graph

3 Commits

Author SHA1 Message Date
Laurent Mazare
0ace420e66 Flash attention without padding (varlen). (#281) 
* Expose the seqlen variable for flash-attn without padding.

* Fix the batched call.

* Adapt for the varlen variant.

* No need to set the batch strides when in varlen mode.

* Add a test (disabled at the moment).

* Get the test to work properly.
 2023-07-31 09:45:39 +01:00
Laurent Mazare
3eb2bc6d07 Softmax numerical stability. (#267) 
* Softmax numerical stability.

* Fix the flash-attn test.
 2023-07-28 13:13:01 +01:00
Laurent Mazare
4f92420132 Add some flash attn test (#253) 
* Add some flash-attn test.

* Add the cpu test.

* Fail when the head is not a multiple of 8.

* Polish the flash attention test.
 2023-07-26 20:56:00 +01:00