5 Commits

Author	SHA1	Message	Date
Nicolas Patry	9c4b4f0da0	Metal quantized modifications proposal. ... - Add a device param, wherever needed. - Create new QMetal storage thing that implements QuantizedType. - Update everywhere needed. Fix Python. Fixing examples. Fix: fmt + clippy + stub. Moving everything around. Only missing the actual implems. Fixing everything + adding dequantized kernels. More work. Fixing matmul. Fmt + Clippy Some clippy fixes. Working state. Q2K Metal -> Bugged (also present in GGML). Q4K CPU -> Bugged (present previously, new test catch it). Q5K CPU -> Bugged (present previously). Q8_1 Both -> Never really implemented it seems Q8K metal -> Never implemented in metal Fixing Q2K bug (present in ggml).	2024-01-15 17:42:58 +01:00
Laurent Mazare	59ab6d7832	Quantized version of StableLM. (#1058) ... * Quantized version of StableLM. * Adapt the stable-lm example to support quantizsed. * Use some separate hub repo. * Another repo name tweak.	2023-10-08 15:42:38 +01:00
Laurent Mazare	823fe23f9b	Add flash-attn support for stable-lm. (#1052)	2023-10-07 21:12:54 +01:00
Laurent Mazare	955e00b2e8	Add to the readmes for stable-lm. (#1047)	2023-10-06 21:26:04 +01:00
Laurent Mazare	d5f7267087	Add the stable-lm example. (#1046) ... * Add the stable-lm example. * Get stable-lm to generate some proper text.	2023-10-06 19:20:35 +01:00