huggingface/candle
1
0
Fork 0
mirror of https://github.com/huggingface/candle.git synced 2025-06-20 12:06:35 +00:00
Code Issues Packages Projects Releases Wiki Activity

Separate quantized phi-3 implementation. (#2157)

Browse Source 
* Separate quantized phi-3 implementation.

* Integrate the quantized phi3 model.=

* Small fixes, get the generation to work properly.

* Keep the old llama implementation around.

* Change the default.
This commit is contained in:
Laurent Mazare
2024-05-04 10:14:57 +02:00
committed by GitHub
parent 59b18d974e
commit b13a82a438
7 changed files with 323 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
candle-metal-kernels/src/lib.rs
View File

@ -350,7 +350,7 @@ pub fn call_unary_contiguous_tiled(
let pipeline = kernels.load_pipeline(device, Source::Unary, kernel_name.0)?;
let encoder = command_buffer.new_compute_command_encoder();
let tile_size = 2;
let tiles = length.div_ceil(tile_size);
let tiles = (length + tile_size - 1) / tile_size;
encoder.set_compute_pipeline_state(&pipeline);