Add the RWKV model (v5). (#1707)

* Start adding the RWKV model.

* More of the forward step.

* Handle rescaling.

* FeedForward.

* More work on RWKV.

* Better state tracking.

* Finish a first pass on forward.

* Fix the shape mismatches.

* Do not rescale in f32.

* Rename to rwkv-v5.

* Add the new models to the readme.
This commit is contained in:
Laurent Mazare
2024-02-14 10:58:32 +01:00
committed by GitHub
parent 68f7655895
commit 2d5f2a728d
5 changed files with 616 additions and 3 deletions

View File

@ -34,6 +34,7 @@ pub mod quantized_t5;
pub mod qwen2;
pub mod repvgg;
pub mod resnet;
pub mod rwkv_v5;
pub mod segment_anything;
pub mod stable_diffusion;
pub mod stable_lm;