4 Commits

Author	SHA1	Message	Date
Laurent Mazare	ca449f9ee1	Add quantized tensors. (#458) ... * Add quantized tensors. * Implement the debug trait for QTensor. * Add the QMatMul custom op.	2023-08-15 22:45:53 +01:00
Laurent Mazare	b8263aa15c	Quantized support for f16 and f32 (#457) ... * Add f32 as a quantized type. * Add f16 as a quantized type too.	2023-08-15 21:09:37 +01:00
Laurent Mazare	e68b2accb4	Split out the quantized file. (#456)	2023-08-15 20:26:27 +01:00
Lukas Kreussel	9e7e6e0288	Add dequantization for ggmls q4_0, q4_1, q5_0, q5_1 and q8_0 (#407) ... * Added dequantization for `q4_0`, `q4_1`, `q5_0`, `q5_1` and `q8_0` * expose `tensor_from_ggml` for external usage * bugfixes & example	2023-08-13 23:22:57 +01:00