huggingface/candle
1
0
Fork 0
mirror of https://github.com/huggingface/candle.git synced 2025-06-18 03:28:50 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,804 Commits 74 Branches 16 Tags
0c49e95dfb5f25a17340ff1b690a4b4f6cd0e2d1
Commit Graph

3 Commits

Author SHA1 Message Date
Jack Shih
918136ba46 add quantized rwkv v5 model (#1743) 
* and quantized rwkv v5 model

* Integrate the quantized rwkv model in the initial example.

---------

Co-authored-by: laurent <laurent.mazare@gmail.com>
 2024-02-25 21:43:40 +01:00
Laurent Mazare
26fe162ab5 Custom tokenizer for rwkv. (#1711) 
* Custom tokenizer for rwkv.

* Custom tokenizer.

* Getting the tokenizer to work.
 2024-02-14 15:11:38 +01:00
Laurent Mazare
2d5f2a728d Add the RWKV model (v5). (#1707) 
* Start adding the RWKV model.

* More of the forward step.

* Handle rescaling.

* FeedForward.

* More work on RWKV.

* Better state tracking.

* Finish a first pass on forward.

* Fix the shape mismatches.

* Do not rescale in f32.

* Rename to rwkv-v5.

* Add the new models to the readme.
 2024-02-14 10:58:32 +01:00