huggingface/candle
1
0
Fork 0
mirror of https://github.com/huggingface/candle.git synced 2025-06-20 12:06:35 +00:00
Code Issues Packages Projects Releases Wiki Activity

Fix sigmoid gradient calculation and move sigmoid into a specialized op (#2114)

Browse Source 
* add sigmoid op

* small fix

* add as a method on `Tensor`

* implement gradient calculation for sigmoid

* add sigmoid tests

* we should have a specialized op for this

* fix clippy

* fix clippy 2

* Revert all previous commits in favor of a `CustomOp` based solution

* use `CustomOp1` implementation

* fix rustfmt

* experimental add metal impl

* add cuda kernel impl

* fix fmt

* Add a test + reduce some cuda duplication.

---------

Co-authored-by: laurent <laurent.mazare@gmail.com>
This commit is contained in:
MilkFather
2024-04-29 17:04:43 +08:00
committed by GitHub
parent ed7b99f525
commit 3bbb88fcb4
6 changed files with 214 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
candle-metal-kernels/src/lib.rs
View File

@ -129,7 +129,7 @@ macro_rules! ops{
pub mod unary {
ops!(
cos, sin, exp, sqr, sqrt, neg, log, gelu, abs, ceil, floor, relu, round, erf, gelu_erf,
tanh, recip, silu, sign
tanh, recip, silu, sign, sigmoid
);
}
pub mod binary {