15 Commits

Author SHA1 Message Date
7bd0faba75 Add support for accelerate in the pyo3 bindings. (#1167) 2023-10-24 06:34:37 +01:00
904bbdae65 Make the Python Wrapper more Hackable and simplify Quantization (#1010)
* Some first `Module` implementations

* Add `state_dict` and `load_state_dict` functionality

* Move modules around and create `candle.nn.Linear`

* Add `nn.Embedding` and `nn.LayerNorm`

* Add BERT implementation

* Batch q-matmul

* Automatically dequantize `QTensors` if a `Tensor` is expected

* Add Module `.to()`, `.cuda()`, `cpu()` and `.type()` functionality

* Unittests for `Module`, `Tensor` and `candle.utils`

* Add `pytorch` like slicing to `Tensor`

* Cleanup and BERT fixes

* `black` formatting + unit-test for `nn.Linear`

* Refactor slicing implementation
2023-10-06 19:01:07 +01:00
03e194123d Add return types to *.pyi stubs (#880)
* Start generating return types

* Finish tensor type hinting

* Add `save_gguf` to `utils`

* Typehint `quant-llama.py`
2023-09-17 22:11:01 +01:00
8658df3485 Generate *.pyi stubs for PyO3 wrapper (#870)
* Begin to generate typehints.

* generate correct stubs

* Correctly include stubs

* Add comments and typhints to static functions

* ensure candle-pyo3 directory

* Make `llama.rope.freq_base` optional

* `fmt`
2023-09-16 17:23:38 +01:00
21109e1983 Recommend using maturin. (#717) 2023-09-02 16:19:35 +01:00
2ed78ab336 Support for quantized tensors in the python api. (#706)
* Add more pyo3 support.

* Add some support for quantized tensors in pyo3.

* Add an arc layer on qmatmul.

* Add the quantized matmul.

* Quantization support.

* More quantization support.

* Test the python quantization.
2023-09-01 15:53:42 +01:00
93cfe5642f Pyo3 dtype (#327)
* Better handling of dtypes in pyo3.

* More pyo3 dtype.
2023-08-06 10:17:43 +01:00
88bd3b604a Add some tensor creation functions to the pyo3 bindings. (#326) 2023-08-06 06:50:33 +01:00
fbfe74caab Preliminary pyo3 support for device. 2023-07-02 20:42:55 +01:00
78871ffe38 Add dtype support. 2023-07-02 20:12:26 +01:00
d38897461b Add to the example. 2023-07-02 07:37:17 +01:00
dfe197f791 Handle more input types to create tensors. 2023-07-02 07:19:46 +01:00
fbbde5b02c Add some binary operators. 2023-07-01 21:27:35 +01:00
42d1a52d01 Add two methods. 2023-07-01 20:55:15 +01:00
ebb0fedf14 Very simple pyo3 bindings for candle. 2023-07-01 20:36:44 +01:00