Laurent Mazare cd29c7ccd4 More ggml cuda kernels (#1977) ...

* Add more cuda kernels for quantized matmul.

* Add the vec-dot bits.

* Expose the quantized matmul-vec kernels.

* Also include the quantize-q8-1 kernel.

* Glue code for the q8-1 quantization.

* mm-vec product via q8-1 quantization.

* Add a test.

* Add a mm test.

* Get the test to return some sensible results.

* Also test dmmv.

* Fix the launch params.

* Allow for tweaking the force_dmmv parameter while it's experimental.

2024-04-01 00:15:48 +02:00

..

bert

Speed up bert with approx gelu (#1410)

2023-12-06 17:46:37 +01:00

bigcode

Adapt more examples to the updated safetensor api. (#947)

2023-09-23 21:26:03 +01:00

blip

Quantized GGUF style (#1523)

2024-01-17 10:27:58 +01:00

chatglm

Add the custom tokenizer. (#1686)

2024-02-09 17:36:50 +01:00

clip

CLIP model implementation with example (#1950)

2024-03-28 13:44:12 +01:00

convmixer

Move the image tensor to the appropriate device. (#1856)

2024-03-16 22:25:46 +01:00

convnext

Move the image tensor to the appropriate device. (#1856)

2024-03-16 22:25:46 +01:00

custom-ops

Cuda acceleration for quantized model. (#1754)

2024-02-25 18:11:47 +01:00

dinov2

Move the image tensor to the appropriate device. (#1856)

2024-03-16 22:25:46 +01:00

distilbert

Distibert (#1366)

2023-11-24 15:09:14 +00:00

efficientnet

Move the image tensor to the appropriate device. (#1856)

2024-03-16 22:25:46 +01:00

efficientvit

Move the image tensor to the appropriate device. (#1856)

2024-03-16 22:25:46 +01:00

encodec

Microphone support for the encodec example. (#1866)

2024-03-18 11:19:46 +01:00

falcon

Use the new hub helper function. (#1484)

2023-12-26 09:44:30 +01:00

gemma

Update gemma README (#1843)

2024-03-13 21:41:36 +01:00

jina-bert

Use the hub model file when possible. (#1190)

2023-10-26 20:00:50 +01:00

llama

Make the cache for the llama model explicit too. (#1745)

2024-02-22 12:04:33 +01:00

llama2-c

Explicit caching in llama2.c.

2024-02-22 10:22:03 +01:00

llama_multiprocess

Use the new hub helper function. (#1484)

2023-12-26 09:44:30 +01:00

mamba

Improved mamba model optimized for inference (#1694)

2024-02-11 17:04:57 +01:00

mamba-minimal

Improved mamba model optimized for inference (#1694)

2024-02-11 17:04:57 +01:00

marian-mt

Add a KV cache to marian decoding. (#1226)

2023-10-31 08:47:44 +00:00

metavoice

Quantized version of the metavoice model. (#1824)

2024-03-09 11:06:04 +01:00

mistral

Support more mistral models. (#1927)

2024-03-24 08:04:04 +01:00

mixtral

Use the tokenizer-output-stream in the llama example. (#1715)

2024-02-15 16:47:33 +01:00

mnist-training

Allow for different behavior between training and eval (#1213)

2023-10-29 07:53:09 +01:00

mobileone

Move the image tensor to the appropriate device. (#1856)

2024-03-16 22:25:46 +01:00

moondream

Add options to use local files + specify a custom repo or branch. (#1973)

2024-03-31 09:32:50 +02:00

musicgen

Support more modes in the encodec example. (#1777)

2024-02-28 09:22:33 +01:00

onnx

Update docs to reflect current usage of example (#1610)

2024-02-04 11:59:47 +01:00

phi

Quantized GGUF style (#1523)

2024-01-17 10:27:58 +01:00

quantized

More ggml cuda kernels (#1977)

2024-04-01 00:15:48 +02:00

quantized-t5

Move the tensor-tools binary in a separate crate. (#1969)

2024-03-30 15:49:37 +01:00

qwen

Readme fix. (#1961)

2024-03-28 23:24:46 +01:00

reinforcement-learning

Apply rustfmt. (#1873)

2024-03-18 21:43:31 +01:00

replit-code

Quantized GGUF style (#1523)

2024-01-17 10:27:58 +01:00

repvgg

Move the image tensor to the appropriate device. (#1856)

2024-03-16 22:25:46 +01:00

resnet

Move the image tensor to the appropriate device. (#1856)

2024-03-16 22:25:46 +01:00

rwkv

Fix the model path for rwkv. (#1825)

2024-03-09 11:21:48 +01:00

segformer

Update the image crate + use the re-exported version. (#1893)

2024-03-21 10:56:41 +01:00

segment-anything

Add negative prompts to segment-anything. (#1000)

2023-09-30 06:17:42 +01:00

stable-diffusion

Improve the error message on overlong prompts. (#1908)

2024-03-21 21:08:07 +01:00

stable-lm

Avoid copying the data on squeeze and unsqueeze. (#1884)

2024-03-20 13:04:36 +01:00

starcoder2

Add the StarCoder2 model. (#1779)

2024-02-28 21:02:41 +01:00

t5

Add some examples using the MT5 variants. (#1963)

2024-03-29 18:09:29 +01:00

trocr

docs: add trocr examples (#1692)

2024-02-10 16:14:50 +01:00

vgg

Move the image tensor to the appropriate device. (#1856)

2024-03-16 22:25:46 +01:00

vit

Move the image tensor to the appropriate device. (#1856)

2024-03-16 22:25:46 +01:00

whisper

whisper readme (#1899)

2024-03-21 12:54:09 +01:00

whisper-microphone

feat: support microphone whisper streaming (#1678)

2024-02-12 18:01:21 +01:00

wuerstchen

Remove some unusued bits. (#1067)

2023-10-09 19:49:57 +01:00

yi

Fix token generation in bilingual models (non-English outputs) (#1668)

2024-02-06 12:03:53 +01:00

yolo-v3

Fix clippy lints for 1.76. (#1682)

2024-02-08 16:48:47 +01:00

yolo-v8

Update the image crate + use the re-exported version. (#1893)

2024-03-21 10:56:41 +01:00

onnx_basics.rs

[ONNX] Support a couple more ops. (#1284)

2023-11-06 22:44:58 +01:00