candle

mirror of https://github.com/huggingface/candle.git synced 2025-06-15 18:28:24 +00:00

Files

Laurent Mazare e8e33752f4 Sketch a quantized llama using the pyo3 api. (#715 )

* Sketch a quantized llama using the pyo3 api.

* Add more ops.

* Expose a few more functions to use in the quantized model.

* Rope embeddings.

* Get the forward pass to work.

2023-09-02 11:26:05 +01:00

src

Sketch a quantized llama using the pyo3 api. (#715 )

2023-09-02 11:26:05 +01:00

build.rs

Fix the pyo3 build for macos. (#324 )

2023-08-05 14:53:57 +01:00

Cargo.toml

Sketch a quantized llama using the pyo3 api. (#715 )

2023-09-02 11:26:05 +01:00

quant-llama.py

Sketch a quantized llama using the pyo3 api. (#715 )

2023-09-02 11:26:05 +01:00

README.md

Fix the pyo3 build for macos. (#324 )

2023-08-05 14:53:57 +01:00

test.py

Support for quantized tensors in the python api. (#706 )

2023-09-01 15:53:42 +01:00

README.md

From the top level directory run the following for linux.

cargo build --profile=release-with-debug --package candle-pyo3 && cp -f ./target/release-with-debug/libcandle.so candle.so
PYTHONPATH=. python3 candle-pyo3/test.py
```bash

  Or for macOS users:
```bash
cargo build --profile=release-with-debug --package candle-pyo3 && cp -f ./target/release-with-debug/libcandle.dylib candle.so
PYTHONPATH=. python3 candle-pyo3/test.py