8 Commits

Author	SHA1	Message	Date
Laurent Mazare	56c9d3ee7b	Fix the model path for rwkv. (#1825)	2024-03-09 11:21:48 +01:00
Jack Shih	6980774a91	fix rwkv example eos token (#1785)	2024-03-01 10:22:28 +01:00
Laurent Mazare	64d4038e4f	Mention rwkv v6 in the readmes. (#1784)	2024-03-01 08:58:30 +01:00
Jack Shih	b485e4b6ee	add models of rwkv v6 and quantized rwkv v6 (#1781) ... * add models of rwkv v6 and quantized rwkv v6 * fix ci clippy fail	2024-03-01 08:37:56 +01:00
Jack Shih	918136ba46	add quantized rwkv v5 model (#1743) ... * and quantized rwkv v5 model * Integrate the quantized rwkv model in the initial example. --------- Co-authored-by: laurent <laurent.mazare@gmail.com>	2024-02-25 21:43:40 +01:00
Laurent Mazare	058a910d0e	Add a readme for rwkv. (#1712)	2024-02-14 15:31:33 +01:00
Laurent Mazare	26fe162ab5	Custom tokenizer for rwkv. (#1711) ... * Custom tokenizer for rwkv. * Custom tokenizer. * Getting the tokenizer to work.	2024-02-14 15:11:38 +01:00
Laurent Mazare	2d5f2a728d	Add the RWKV model (v5). (#1707) ... * Start adding the RWKV model. * More of the forward step. * Handle rescaling. * FeedForward. * More work on RWKV. * Better state tracking. * Finish a first pass on forward. * Fix the shape mismatches. * Do not rescale in f32. * Rename to rwkv-v5. * Add the new models to the readme.	2024-02-14 10:58:32 +01:00