000487c36f
Add a python function to save as safetensors. ( #740 )
2023-09-04 20:32:14 +01:00
20512ba408
Return the metadata in the gguf pyo3 bindings. ( #729 )
...
* Return the metadata in the gguf pyo3 bindings.
* Read the metadata in the quantized llama example.
* Get inference to work on gguf files.
2023-09-04 07:07:00 +01:00
84d003ff53
Handle arbitrary shapes in Tensor::new. ( #718 )
2023-09-02 19:59:21 +01:00
ad796eb4be
More quantized llama in python. ( #716 )
...
* More quantized llama in python.
* Expose a couple more functions.
* Apply the last layer.
* Use the vocab from the ggml files.
2023-09-02 13:41:48 +01:00
e8e33752f4
Sketch a quantized llama using the pyo3 api. ( #715 )
...
* Sketch a quantized llama using the pyo3 api.
* Add more ops.
* Expose a few more functions to use in the quantized model.
* Rope embeddings.
* Get the forward pass to work.
2023-09-02 11:26:05 +01:00
1e5b2cc1d5
Add some quantized functions to pyo3. ( #708 )
2023-09-01 19:45:36 +02:00
2ed78ab336
Support for quantized tensors in the python api. ( #706 )
...
* Add more pyo3 support.
* Add some support for quantized tensors in pyo3.
* Add an arc layer on qmatmul.
* Add the quantized matmul.
* Quantization support.
* More quantization support.
* Test the python quantization.
2023-09-01 15:53:42 +01:00
237323c2bc
Cleanup the pyo3 setup. ( #705 )
2023-09-01 14:26:18 +01:00
e21c686cdc
Fixes for clippy 1.72. ( #587 )
2023-08-24 17:46:17 +01:00
9a5c7db91a
Add support for i64 ( #563 )
...
* Add the i64 dtype.
* Adapt the cuda kernels.
2023-08-23 10:42:19 +01:00
93cfe5642f
Pyo3 dtype ( #327 )
...
* Better handling of dtypes in pyo3.
* More pyo3 dtype.
2023-08-06 10:17:43 +01:00
88bd3b604a
Add some tensor creation functions to the pyo3 bindings. ( #326 )
2023-08-06 06:50:33 +01:00
2bfa791336
Use the same default as pytorch for sum. ( #164 )
2023-07-13 21:32:32 +01:00
50b0946a2d
Tensor mutability ( #154 )
...
* Working towards tensor mutability.
* Use a ref-cell to provide tensor mutability.
2023-07-13 11:04:40 +01:00
5b0ee2e0ba
Get cuda to work on pyo3.
2023-07-02 21:04:11 +01:00
fbfe74caab
Preliminary pyo3 support for device.
2023-07-02 20:42:55 +01:00
bdb257ceab
Add the tensor function.
2023-07-02 20:15:50 +01:00
78871ffe38
Add dtype support.
2023-07-02 20:12:26 +01:00
5b8c6764b0
Add matmul/where_cond.
2023-07-02 07:34:14 +01:00
9a9858bbe0
Expose a couple more ops.
2023-07-02 07:30:00 +01:00
dfe197f791
Handle more input types to create tensors.
2023-07-02 07:19:46 +01:00
4a28dcf828
Rename the method.
2023-07-02 07:08:11 +01:00
c62cb73a7f
Support higher order shapes for conversions.
2023-07-02 07:07:22 +01:00
fa58c7643d
Add a trait to avoid repeating the dtype matching.
2023-07-02 06:58:10 +01:00
2370b1675d
More pyo3.
2023-07-01 22:15:58 +01:00
86df4ad79c
Get shape to return a tuple.
2023-07-01 21:34:38 +01:00
fbbde5b02c
Add some binary operators.
2023-07-01 21:27:35 +01:00
42d1a52d01
Add two methods.
2023-07-01 20:55:15 +01:00
52db2a6849
Apply rustfmt.
2023-07-01 20:37:28 +01:00
ebb0fedf14
Very simple pyo3 bindings for candle.
2023-07-01 20:36:44 +01:00
