7 Commits

Author	SHA1	Message	Date
Lukas Kreussel	904bbdae65	Make the Python Wrapper more Hackable and simplify Quantization (#1010) ... * Some first `Module` implementations * Add `state_dict` and `load_state_dict` functionality * Move modules around and create `candle.nn.Linear` * Add `nn.Embedding` and `nn.LayerNorm` * Add BERT implementation * Batch q-matmul * Automatically dequantize `QTensors` if a `Tensor` is expected * Add Module `.to()`, `.cuda()`, `cpu()` and `.type()` functionality * Unittests for `Module`, `Tensor` and `candle.utils` * Add `pytorch` like slicing to `Tensor` * Cleanup and BERT fixes * `black` formatting + unit-test for `nn.Linear` * Refactor slicing implementation	2023-10-06 19:01:07 +01:00
Lukas Kreussel	03e194123d	Add return types to *.pyi stubs (#880) ... * Start generating return types * Finish tensor type hinting * Add `save_gguf` to `utils` * Typehint `quant-llama.py`	2023-09-17 22:11:01 +01:00
Lukas Kreussel	8658df3485	Generate *.pyi stubs for PyO3 wrapper (#870) ... * Begin to generate typehints. * generate correct stubs * Correctly include stubs * Add comments and typhints to static functions * ensure candle-pyo3 directory * Make `llama.rope.freq_base` optional * `fmt`	2023-09-16 17:23:38 +01:00
Laurent Mazare	20512ba408	Return the metadata in the gguf pyo3 bindings. (#729) ... * Return the metadata in the gguf pyo3 bindings. * Read the metadata in the quantized llama example. * Get inference to work on gguf files.	2023-09-04 07:07:00 +01:00
Laurent Mazare	21109e1983	Recommend using maturin. (#717)	2023-09-02 16:19:35 +01:00
Laurent Mazare	ad796eb4be	More quantized llama in python. (#716) ... * More quantized llama in python. * Expose a couple more functions. * Apply the last layer. * Use the vocab from the ggml files.	2023-09-02 13:41:48 +01:00
Laurent Mazare	e8e33752f4	Sketch a quantized llama using the pyo3 api. (#715) ... * Sketch a quantized llama using the pyo3 api. * Add more ops. * Expose a few more functions to use in the quantized model. * Rope embeddings. * Get the forward pass to work.	2023-09-02 11:26:05 +01:00