huggingface/candle
1
0
Fork 0
mirror of https://github.com/huggingface/candle.git synced 2025-06-19 19:58:35 +00:00
Code Issues Packages Projects Releases Wiki Activity

Llama more training (#297)

Browse Source 
* Rework the var-builder to handle initializations.

* Add some helper functions for layer creation.

* Improve the layer initializations.

* Get initialized variables.

* Precompute the rot embeddings when training lamas.
This commit is contained in:
Laurent Mazare
2023-08-01 19:53:41 +01:00
committed by GitHub
parent a27239f3d9
commit ff876c2103
10 changed files with 238 additions and 163 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
candle-nn/src/lib.rs
View File

@ -15,9 +15,9 @@ pub mod vision;
pub use activation::Activation;
pub use conv::{Conv1d, Conv1dConfig};
pub use embedding::Embedding;
pub use embedding::{embedding, Embedding};
pub use init::Init;
pub use layer_norm::LayerNorm;
pub use linear::Linear;
pub use layer_norm::{layer_norm, LayerNorm};
pub use linear::{linear, linear_no_bias, Linear};
pub use optim::SGD;
pub use var_builder::VarBuilder;
pub use var_builder::{VarBuilder, VarMap};