huggingface/candle
1
0
Fork 0
mirror of https://github.com/huggingface/candle.git synced 2025-06-20 04:00:28 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update the Phi model to use the updated architecture. (#1580)

Browse Source 
* Update the Phi model to use the updated architecture.

* Add more of the phi model.

* Repeat KV + caching.

* Apply the rotary embeddings.

* Add support for the new phi model in the phi example.

* Fix a couple glitches.

* Fix a couple more glitches.
This commit is contained in:
Laurent Mazare
2024-01-13 17:38:27 +01:00
committed by GitHub
parent a46864bd56
commit 539ead927a
4 changed files with 402 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
candle-nn/src/activation.rs
View File

@ -6,6 +6,7 @@ use serde::Deserialize;
pub enum Activation {
#[default]
Gelu,
#[serde(alias = "gelu_new")]
NewGelu,
Relu,
Relu2,