mirror of
https://github.com/huggingface/candle.git
synced 2025-06-16 02:38:10 +00:00
19e52e5007
* init t5 wasm model * split workers for each model * clean up * add some ui * readme * index * typo * remove cache param, clear_kv_cache * add max_length as param * add model tasks option to ui * add method to load quantized gguf from buffer * Add quantized wasm module * add quantized models to UI, dynamic import wasms * link to quantized * fix copy * fix ModelEncoder * fix README.md
Running T5 with Candle and WASM
Here, we provide two examples of how to run Bert using a Candle-compiled WASM binary and runtime.
Vanilla JS and WebWorkers
To build and test the UI made in Vanilla JS and WebWorkers, first we need to build the WASM library:
sh build-lib.sh
This will bundle the library under ./build and we can import it inside our WebWorker like a normal JS module:
import init, { ModelConditionalGeneration, ModelEncoder } from "./build/m.js";
For the quantized version, we need to import the quantized module:
import init, { ModelConditionalGeneration, ModelEncoder } from "./build/m-quantized.js";
The full example can be found under ./index.html. All needed assets are fetched from the web, so no need to download anything.
Finally, you can preview the example by running a local HTTP server. For example:
python -m http.server
Then open http://localhost:8000/index.html in your browser.