huggingface/candle
1
0
Fork 0
mirror of https://github.com/huggingface/candle.git synced 2025-06-17 11:08:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

Implement group-norm. (#334)

Browse Source 
* Implement group-norm.

* Add some testing for group-norm.
This commit is contained in:
Laurent Mazare
2023-08-07 07:53:05 +02:00
committed by GitHub
parent 2c9f605976
commit 5bb2fce998
5 changed files with 150 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
candle-nn/src/ops.rs
View File

@ -34,5 +34,11 @@ pub fn log_softmax<D: candle::shape::Dim>(xs: &Tensor, d: D) -> Result<Tensor> {
}
pub fn silu(xs: &Tensor) -> Result<Tensor> {
// TODO: Should we have a specialized op for this?
xs / (xs.neg()?.exp()? + 1.0)?
}
pub fn sigmoid(xs: &Tensor) -> Result<Tensor> {
// TODO: Should we have a specialized op for this?
(xs.neg()?.exp()? + 1.0)?.recip()
}