c78ce76501
Add a simple Module trait and implement it for the various nn layers ( #500 )
...
* Start adding the module trait.
* Use the module trait.
* Implement module for qmatmul.
2023-08-18 09:38:22 +01:00
13401df4d1
Add an abstract type for RmsNorm. ( #499 )
2023-08-18 08:52:14 +01:00
26fd37b348
Use the main branch of the HF repo where possible. ( #498 )
...
* Use the main branch of the HF repo where possible.
* And add the large model.
2023-08-18 08:18:30 +01:00
f056dcab21
Add medium model ( #497 )
2023-08-18 08:08:59 +01:00
557b2c28dd
Q6K quantization ( #495 )
...
* Print the detected arch options.
* Add the q6k quantization.
* Add a currently broken test.
* Bugfix.
* Bugfix.
* Another bugfix.
* Another bugfix + get the test to work.
2023-08-17 22:22:57 +01:00
3164cd24fa
Replicate the sot-token logic from the Python implementation more acc… ( #491 )
...
* Replicate the sot-token logic from the Python implementation more accurately.
* Add a flag to control the timestamp mode.
2023-08-17 16:59:36 +01:00
5f30c1e1e0
Add the whisper small model. ( #490 )
2023-08-17 15:48:34 +01:00
ad7c53953b
Add a verbose-prompt mode, similar to llama.cpp. ( #489 )
2023-08-17 15:26:44 +01:00
5d99026fd2
F16 support for stable diffusion ( #488 )
...
* F16 support for stable diffusion.
* Keep the attention bits in F32.
* Keep more of the attention bits in F32.
* More mixed precision support.
2023-08-17 13:48:56 +01:00
c3176f0dfb
Flash-attention support in stable diffusion ( #487 )
...
* Add flash-attention for the stable-diffusion example.
* Change the dtype.
* Silly fix.
* Another fix.
* Revert the dtype back to the query dtype after apply flash-attn.
2023-08-17 12:16:40 +01:00
03be33eea4
Relax the requirements on CustomOp. ( #486 )
...
* Relax the requirements on CustomOp.
* Simplify the custom-ops when no backward is required.
2023-08-17 11:12:05 +01:00
d32e8199cd
Layer norm tweaks ( #482 )
...
* Add some options to make layer-norm more configurable.
* Add the rms-norm variant.
* Replace the RmsNorm with the shared bits.
2023-08-17 10:07:13 +01:00
d99cac3ec3
Move the avx specific bits to a separate file. ( #481 )
2023-08-17 09:01:06 +01:00
098909de40
Add vecdot for q6k-q8k. ( #476 )
...
* Add vecdot for q6k-q8k.
* Add some testing for q8k.
* Use QMatMul for the output layer.
2023-08-16 20:59:40 +01:00
c5f45887dc
Add some tracing to the quantized example. ( #473 )
2023-08-16 18:49:08 +01:00
fa4590d7fd
Merge pull request #469 from huggingface/fix_llama_v1
...
Fixing llamav1
2023-08-16 17:47:40 +02:00
2e206e269d
Add the model argument. ( #471 )
2023-08-16 16:41:06 +01:00
575e88a999
Add a quantized test that use negative values. ( #470 )
...
* Add a quantized test that use negative values.
* Add a default tokenizer.
2023-08-16 16:32:58 +01:00
a9101700b6
Add a kv-cache to the quantized llama example. ( #466 )
...
* Add a kv-cache to the quantized llama example.
* Also print the prompt.
* Bugfix in q6k dequantizing.
* Another bugfix.
2023-08-16 14:28:42 +01:00
102fa4c2e3
Fixing llamav1
2023-08-16 14:53:29 +02:00
3071134788
Get the ggml based llama to generate some text. ( #464 )
...
* Add more stats to the ggml example.
* Build a quantized model from the file content.
* Move the tensor retrieval in the main crate.
* Start adding the forward pass.
* Add more to the forward pass of the quantized llama.
* Apply the attention layers.
* Add the sampling loop.
* Get the sampling loop to work.
* Minor tweak.
* Add a quantize/dequantize test.
* Bugfix.
* Add a comment + swap the order.
* Bugfixes.
2023-08-16 12:41:07 +01:00
33c882ea74
Clippy.
2023-08-16 10:41:00 +02:00
76804730c6
Using the real config from the hub when available.
2023-08-16 10:36:01 +02:00
ca449f9ee1
Add quantized tensors. ( #458 )
...
* Add quantized tensors.
* Implement the debug trait for QTensor.
* Add the QMatMul custom op.
2023-08-15 22:45:53 +01:00
b8263aa15c
Quantized support for f16 and f32 ( #457 )
...
* Add f32 as a quantized type.
* Add f16 as a quantized type too.
2023-08-15 21:09:37 +01:00
e68b2accb4
Split out the quantized file. ( #456 )
2023-08-15 20:26:27 +01:00
5b1690fffa
Tweak the llama example. ( #450 )
2023-08-15 12:18:20 +01:00
3cc87058b7
Support local weights & dynamic outputs ( #447 )
...
* Support local weights & dynamic outputs
* Revise as suggested
* Cargo code format
2023-08-15 11:51:57 +01:00
c84883ecf2
Add a cuda kernel for upsampling. ( #441 )
...
* Add a cuda kernel for upsampling.
* Update for the latest tokenizers version.
2023-08-14 13:12:17 +01:00
9e7e6e0288
Add dequantization for ggmls q4_0, q4_1, q5_0, q5_1 and q8_0 ( #407 )
...
* Added dequantization for `q4_0`, `q4_1`, `q5_0`, `q5_1` and `q8_0`
* expose `tensor_from_ggml` for external usage
* bugfixes & example
2023-08-13 23:22:57 +01:00
8bd2b22b33
Optimize the logit computations in the whisper example. ( #434 )
2023-08-13 22:00:13 +01:00
9af438ac1b
Track the conv2d operations in stable-diffusion. ( #431 )
...
* Track the conv2d operations in stable-diffusion.
* Add more tracing to stable-diffusion.
* Also trace the resnet bits.
* Trace the attention blocks.
* Also trace the attention inner part.
* Small tweak.
2023-08-13 15:58:26 +01:00
b1ff78f762
Allow using accelerate with stable-diffusion. ( #430 )
2023-08-13 14:14:20 +01:00
6d694554b8
Support longer sequences in language detection. ( #428 )
2023-08-13 13:16:15 +01:00
9aca398a4f
More accelerate optimizations ( #427 )
...
* Add more tracing to the whisper example.
* Support accelerate in more examples.
* Use accelerate for pointwise functions.
* Use accelerate for binary operations too.
* Bugfix for binary operation: use the rhs before the lhs.
2023-08-13 12:53:34 +01:00
60cd1551ca
Add a KV cache to whisper. ( #426 )
2023-08-12 21:17:08 +01:00
a0908d212c
Add a -language argument. ( #425 )
2023-08-12 17:08:40 +01:00
0741ebbd51
More multilingual support for whisper. ( #419 )
...
* More multilingual support for whisper.
* Use the language token appropriately.
2023-08-12 15:32:52 +01:00
0c3f109faa
Basic multilingual support for whisper ( #417 )
...
* Multi-lingual support for whisper.
* Avoid hardcoding the token names.
* More multi-lingual support.
* Remove the todo.
2023-08-12 11:23:04 +01:00
1d0157bbc4
Stable diffusion: retrieve the model files from the HF hub. ( #414 )
...
* Retrieve the model files from the HF hub in the stable diffusion example.
* Add to the readme.
2023-08-11 18:57:06 +01:00
91dbf907d3
Add more whisper variants. ( #413 )
2023-08-11 17:33:55 +01:00
906c0f3eb5
Remove the checkpoint conversion script. ( #405 )
...
* Remove the checkpoint conversion script.
* Remove references to the script.
2023-08-11 05:59:48 +01:00
80f0482f26
Fix the stable-diffusion vae. ( #398 )
...
* Fix the stable-diffusion vae.
* Fix for saving images.
2023-08-10 18:24:31 +01:00
385f0d261c
Normalize embeddings in the bert example. ( #390 )
2023-08-10 13:05:55 +01:00
c3a0761e62
Add some tracing to the whisper example. ( #375 )
2023-08-09 19:58:36 +01:00
a3b1699409
Embed the mel filters in the whisper binary. ( #373 )
2023-08-09 18:27:26 +01:00
3a62aee91f
Write the generated images using the image crate. ( #363 )
...
* Use the image crate to write the generated images.
* Make the dependency optional.
2023-08-09 15:26:44 +01:00
be21d7e75a
Fix the padding used in stable diffusion. ( #362 )
2023-08-09 13:23:59 +01:00
89d3926c9b
Fixes for the stable diffusion example. ( #342 )
...
* Fixes for the stable diffusion example.
* Bugfix.
* Another fix.
* Fix for group-norm.
* More fixes to get SD to work.
2023-08-08 14:57:09 +01:00
fc265d9dcf
Some CLIP fixes for stable diffusion. ( #338 )
...
* Some CLIP fixes for stable diffusion.
* Add the avg-pool2d operation on cpu.
2023-08-07 18:31:45 +01:00
