6 Commits

Author	SHA1	Message	Date
Nicolas Patry	9c4b4f0da0	Metal quantized modifications proposal. ... - Add a device param, wherever needed. - Create new QMetal storage thing that implements QuantizedType. - Update everywhere needed. Fix Python. Fixing examples. Fix: fmt + clippy + stub. Moving everything around. Only missing the actual implems. Fixing everything + adding dequantized kernels. More work. Fixing matmul. Fmt + Clippy Some clippy fixes. Working state. Q2K Metal -> Bugged (also present in GGML). Q4K CPU -> Bugged (present previously, new test catch it). Q5K CPU -> Bugged (present previously). Q8_1 Both -> Never really implemented it seems Q8K metal -> Never implemented in metal Fixing Q2K bug (present in ggml).	2024-01-15 17:42:58 +01:00
Laurent Mazare	86e7d539d2	Add the quantized mpt model. (#1123) ... * Add the quantized mpt model. * Support the quantized model for replit-code.	2023-10-18 16:29:38 +01:00
Laurent Mazare	63c204c79e	Add a mention to the replit-code model in the readme. (#1121)	2023-10-18 11:27:23 +01:00
Laurent Mazare	767a6578f1	MPT alibi fixes. (#1120) ... * MPT alibi fixes. * Some more fixes. * Finally get the model to return some sensible outputs. * Add a readme.	2023-10-18 10:58:05 +01:00
Laurent Mazare	2cd745a97c	MPT fixes. (#1117) ... * MPT fixes. * Another couple fixes. * Another shape fix.	2023-10-17 21:53:31 +01:00
Laurent Mazare	a72b50e2c0	Build alibi bias. (#1115) ... * Build alibi bias. * Apply the alibi attention bias. * Add the replit-code example.	2023-10-17 20:41:37 +01:00