huggingface/candle
1
0
Fork 0
mirror of https://github.com/huggingface/candle.git synced 2025-06-16 18:48:51 +00:00
Code Issues Packages Projects Releases Wiki Activity

Simd128 version of the q2k-q8k vecdot product. (#1011)

Browse Source 
* Sketch the simd128 version of q2k vecdot.

* Use a single accumulator.

* Simdify the q2k-q8k vecdot product.

* Cosmetic change.
This commit is contained in:
Laurent Mazare
2023-09-30 20:12:41 +01:00
committed by GitHub
parent deee7612da
commit 4e55aaa51f
4 changed files with 80 additions and 50 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
candle-core/tests/quantized_tests.rs
View File

@ -514,7 +514,7 @@ fn ggml_matmul_error_test<T: GgmlType>() -> Result<()> {
let ggml_error = ggml_reference_matmul_error(T::DTYPE)?;
if error > GGML_MAX_DOT_PRODUCT_ERROR {
if !error.is_finite() || error > GGML_MAX_DOT_PRODUCT_ERROR {
candle_core::bail!(
"Dot product error {} exceeds max error {}",
error,