3 Commits

Author	SHA1	Message	Date
Laurent Mazare	ab9019425a	Make the metal sdpa tests deterministic. (#2750)	2025-01-28 09:05:24 +01:00
Laurent Mazare	1a32107fab	Add a few metal gather ops. (#2740) ... * Add a few metal gather ops. * Fix some compilation issues. * Adjust the tolerance.	2025-01-25 23:31:03 +01:00
Eric Buehler	e2b6b367fa	Add some fast Metal MLX SDPA kernels (#2584) ... * Add some fast Metal MLX SDPA kernels (#32) * Sketch the sdpa kernel * Add full sdpa kernel, * Add test * Add vectorized kernel for decoding * Update tests * Add some docs * Fix sdpa_vector names * Add softcapping for vectorized sdpa * Add softcapping for full sdpa * Add support for head dim 32, 96, 256 * Add support for head dim 32, 96, 256 * Update docs * Add update notice * Clippy and format * Conditional compilation for bf16 * Use it in quantized llama * Some review comments * Use set_params! * Remove unused * Remove feature * Fix metal sdpa for v stride * Remove comma * Add the dim method to layout and shape. --------- Co-authored-by: Laurent <laurent.mazare@gmail.com>	2024-11-05 09:28:00 +01:00