huggingface/candle
1
0
Fork 0
mirror of https://github.com/huggingface/candle.git synced 2025-06-18 19:47:12 +00:00
Code Issues Packages Projects Releases Wiki Activity

Simd128 vec-dot for q4_0. (#974)

Browse Source 
* Simd128 vec-dot for q4_0.

* Bugfix.

* Add wasm tests.

* Bugfix for the q40 vecdot.

* More quantization tests.
This commit is contained in:
Laurent Mazare
2023-09-27 14:15:30 +01:00
committed by GitHub
parent e59784e353
commit 667f01c173
8 changed files with 253 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
candle-core/src/quantized/k_quants.rs
View File

@ -225,6 +225,9 @@ impl GgmlType for BlockQ4_0 {
#[cfg(target_feature = "neon")]
return super::neon::vec_dot_q4_0_q8_0(n, xs, ys);
#[cfg(target_feature = "simd128")]
return super::simd128::vec_dot_q4_0_q8_0(n, xs, ys);
let qk = QK8_0;
let nb = n / qk;
if n % QK8_0 != 0 {