candle

mirror of https://github.com/huggingface/candle.git synced 2025-06-18 19:47:12 +00:00

Files

Nicolas Patry 9c4b4f0da0 Metal quantized modifications proposal.

- Add a device param, wherever needed.
- Create new QMetal storage thing that implements QuantizedType.
- Update everywhere needed.

Fix Python.

Fixing examples.

Fix: fmt + clippy + stub.

Moving everything around.

Only missing the actual implems.

Fixing everything + adding dequantized kernels.

More work.

Fixing matmul.

Fmt + Clippy

Some clippy fixes.

Working state.

Q2K Metal -> Bugged (also present in GGML).
Q4K CPU -> Bugged (present previously, new test catch it).
Q5K CPU -> Bugged (present previously).
Q8_1 Both -> Never really implemented it seems
Q8K metal -> Never implemented in metal

Fixing Q2K bug (present in ggml).

2024-01-15 17:42:58 +01:00

src

Simd128 vec-dot for q4_0. (#974 )

2023-09-27 14:15:30 +01:00

tests

Metal quantized modifications proposal.

2024-01-15 17:42:58 +01:00

Cargo.toml

Simplifying our internal cargo dependencies. (#1529 )

2024-01-07 12:04:14 +01:00

README.md

Simd128 vec-dot for q4_0. (#974 )

2023-09-27 14:15:30 +01:00

webdriver.json

Simd128 vec-dot for q4_0. (#974 )

2023-09-27 14:15:30 +01:00

README.md

Run the tests with:

RUST_LOG=wasm_bindgen_test_runner wasm-pack test --chrome --headless

Or:

wasm-pack test --chrome

If you get an "invalid session id" failure in headless mode, check that logs and it may well be that your ChromeDriver is not at the same version as your browser.