huggingface/candle
1
0
Fork 0
mirror of https://github.com/huggingface/candle.git synced 2025-06-18 19:47:12 +00:00
Code Issues Packages Projects Releases Wiki Activity

Simd128 q2k vecdot (#982)

Browse Source 
* Sketch the simd128 version of q2k vecdot.

* Use a single accumulator.
This commit is contained in:
Laurent Mazare
2023-09-28 12:16:35 +01:00
committed by GitHub
parent 5e1c595e00
commit 25657804ef
2 changed files with 57 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
candle-core/src/quantized/k_quants.rs
View File

@ -687,6 +687,9 @@ impl GgmlType for BlockQ2K {
#[cfg(target_feature = "neon")]
return super::neon::vec_dot_q2k_q8k(n, xs, ys);
#[cfg(target_feature = "simd128")]
return super::simd128::vec_dot_q2k_q8k(n, xs, ys);
if n % QK_K != 0 {
crate::bail!("vec_dot_q2k_q8k: {n} is not divisible by {QK_K}")
}