huggingface/candle
1
0
Fork 0
mirror of https://github.com/huggingface/candle.git synced 2025-06-16 10:38:54 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
17313a4226a6c6bde444d28b4be4f0f96d155be7
candle/candle-examples/examples/onnx-llm/README.md
Kyle Birnbaum cd7b877d6b candle-onnx: Implement Trilu and ScatterND ops (#2952) 
* onnx attention

* setup an example, adding and fixing onnx ops bit by bit

* model working, output is garbage data

* trilu working

* close but not quite, Issues still with scatterND

* closer but the outputs are still slightly wrong

* added tests for trilu and scatterND

* lint

* readme

* clippy

* removed unnessisary comments

* changed device selection, took hyperparameters from model config
2025-05-30 07:36:09 +02:00

295 B
Raw Blame History

Using ONNX models in Candle

This example demonstrates how to run ONNX based LLM models in Candle.

This script only implements SmolLM-135M right now.

You can run the examples with following commands:

cargo run --example onnx-llm --features onnx