candle

mirror of https://github.com/huggingface/candle.git synced 2025-06-15 02:16:37 +00:00

Files

Laurent Mazare ad796eb4be More quantized llama in python. (#716 )

* More quantized llama in python.

* Expose a couple more functions.

* Apply the last layer.

* Use the vocab from the ggml files.

2023-09-02 13:41:48 +01:00

src

More quantized llama in python. (#716 )

2023-09-02 13:41:48 +01:00

build.rs

Fix the pyo3 build for macos. (#324 )

2023-08-05 14:53:57 +01:00

Cargo.toml

Sketch a quantized llama using the pyo3 api. (#715 )

2023-09-02 11:26:05 +01:00

quant-llama.py

More quantized llama in python. (#716 )

2023-09-02 13:41:48 +01:00

README.md

Fix the pyo3 build for macos. (#324 )

2023-08-05 14:53:57 +01:00

test.py

Support for quantized tensors in the python api. (#706 )

2023-09-01 15:53:42 +01:00

README.md

From the top level directory run the following for linux.

cargo build --profile=release-with-debug --package candle-pyo3 && cp -f ./target/release-with-debug/libcandle.so candle.so
PYTHONPATH=. python3 candle-pyo3/test.py
```bash

  Or for macOS users:
```bash
cargo build --profile=release-with-debug --package candle-pyo3 && cp -f ./target/release-with-debug/libcandle.dylib candle.so
PYTHONPATH=. python3 candle-pyo3/test.py