c78ce76501
Add a simple Module trait and implement it for the various nn layers ( #500 )
...
* Start adding the module trait.
* Use the module trait.
* Implement module for qmatmul.
2023-08-18 09:38:22 +01:00
5d99026fd2
F16 support for stable diffusion ( #488 )
...
* F16 support for stable diffusion.
* Keep the attention bits in F32.
* Keep more of the attention bits in F32.
* More mixed precision support.
2023-08-17 13:48:56 +01:00
c3176f0dfb
Flash-attention support in stable diffusion ( #487 )
...
* Add flash-attention for the stable-diffusion example.
* Change the dtype.
* Silly fix.
* Another fix.
* Revert the dtype back to the query dtype after apply flash-attn.
2023-08-17 12:16:40 +01:00
9af438ac1b
Track the conv2d operations in stable-diffusion. ( #431 )
...
* Track the conv2d operations in stable-diffusion.
* Add more tracing to stable-diffusion.
* Also trace the resnet bits.
* Trace the attention blocks.
* Also trace the attention inner part.
* Small tweak.
2023-08-13 15:58:26 +01:00
b1ff78f762
Allow using accelerate with stable-diffusion. ( #430 )
2023-08-13 14:14:20 +01:00
1d0157bbc4
Stable diffusion: retrieve the model files from the HF hub. ( #414 )
...
* Retrieve the model files from the HF hub in the stable diffusion example.
* Add to the readme.
2023-08-11 18:57:06 +01:00
80f0482f26
Fix the stable-diffusion vae. ( #398 )
...
* Fix the stable-diffusion vae.
* Fix for saving images.
2023-08-10 18:24:31 +01:00
3a62aee91f
Write the generated images using the image crate. ( #363 )
...
* Use the image crate to write the generated images.
* Make the dependency optional.
2023-08-09 15:26:44 +01:00
be21d7e75a
Fix the padding used in stable diffusion. ( #362 )
2023-08-09 13:23:59 +01:00
89d3926c9b
Fixes for the stable diffusion example. ( #342 )
...
* Fixes for the stable diffusion example.
* Bugfix.
* Another fix.
* Fix for group-norm.
* More fixes to get SD to work.
2023-08-08 14:57:09 +01:00
fc265d9dcf
Some CLIP fixes for stable diffusion. ( #338 )
...
* Some CLIP fixes for stable diffusion.
* Add the avg-pool2d operation on cpu.
2023-08-07 18:31:45 +01:00
2345b8ce3f
Skeleton for the avg-pool2d and upsample-nearest2d ops. ( #337 )
...
* Skeleton for the avg-pool2d and upsample-nearest2d ops.
* Preliminary conv2d support.
2023-08-07 16:15:38 +01:00
f53a333ea9
Simple pad support. ( #336 )
...
* Simple pad support.
* Fix the tensor indexing when padding.
2023-08-07 15:24:56 +01:00
5bb2fce998
Implement group-norm. ( #334 )
...
* Implement group-norm.
* Add some testing for group-norm.
2023-08-07 06:53:05 +01:00
141df4ad2b
Main diffusion loop for the SD example. ( #332 )
2023-08-06 21:39:53 +01:00
166bfd5847
Add the recip op + use it in stable-diffusion. ( #331 )
...
* Add the recip unary op.
* Fix the cuda kernel.
* Use the recip op in sigmoid.
2023-08-06 21:14:52 +01:00
1c062bf06b
Add the ddim scheduler. ( #330 )
2023-08-06 20:44:00 +01:00
d34039e352
Add a stable diffusion example ( #328 )
...
* Start adding a stable-diffusion example.
* Proper computation of the causal mask.
* Add the chunk operation.
* Work in progress: port the attention module.
* Add some dummy modules for conv2d and group-norm, get the attention module to compile.
* Re-enable the 2d convolution.
* Add the embeddings module.
* Add the resnet module.
* Add the unet blocks.
* Add the unet.
* And add the variational auto-encoder.
* Use the pad function from utils.
2023-08-06 17:49:43 +01:00
