Laurent Mazare 2d5f2a728d Add the RWKV model (v5). (#1707) ...

* Start adding the RWKV model.

* More of the forward step.

* Handle rescaling.

* FeedForward.

* More work on RWKV.

* Better state tracking.

* Finish a first pass on forward.

* Fix the shape mismatches.

* Do not rescale in f32.

* Rename to rwkv-v5.

* Add the new models to the readme.

2024-02-14 10:58:32 +01:00

..

main.rs

Add the RWKV model (v5). (#1707)

2024-02-14 10:58:32 +01:00