4 Commits

Author	SHA1	Message	Date
Laurent Mazare	1a6043af51	Tweak the VarMap set type. (#1758)	2024-02-25 20:50:08 +01:00
Laurent Mazare	5ebcfeaf0f	Make the r, k, v tensors contiguous. (#1719)	2024-02-16 09:17:35 +01:00
Laurent Mazare	26fe162ab5	Custom tokenizer for rwkv. (#1711) ... * Custom tokenizer for rwkv. * Custom tokenizer. * Getting the tokenizer to work.	2024-02-14 15:11:38 +01:00
Laurent Mazare	2d5f2a728d	Add the RWKV model (v5). (#1707) ... * Start adding the RWKV model. * More of the forward step. * Handle rescaling. * FeedForward. * More work on RWKV. * Better state tracking. * Finish a first pass on forward. * Fix the shape mismatches. * Do not rescale in f32. * Rename to rwkv-v5. * Add the new models to the readme.	2024-02-14 10:58:32 +01:00