60fdab4e17
Detach all grads during backprop. ( #1243 )
...
* Detach all grads during backprop.
* Add an environment variable to select the backprop behavior.
* Update the comment.
2023-11-05 14:07:41 +01:00
928a9d906e
[ONNX] Do not generate values for constants. ( #1272 )
...
* Do not generate values for constants.
* Add an onnx based example using squeezenet.
2023-11-05 11:23:14 +01:00
d1d89bac1f
feat: download cifar dataset parquet files ( #1259 )
2023-11-05 10:55:49 +01:00
39ad840a90
Better tensor initialization in ONNX. ( #1270 )
...
* Better tensor initialization in ONNX.
* MaxPool support.
* Add AvgPool.
* Get the squeezenet example to work.
2023-11-04 22:17:45 +01:00
b5e4f84bed
Refactor the onnx attribute getters. ( #1268 )
...
* Refactor the onnx attribute getters.
* Add get-attr-opt.
* Add support for convolutions.
* Add support for convolutions.
2023-11-04 21:31:48 +01:00
7051fb8098
feat: add backprop for elu ( #1269 )
...
* feat: add backprop for elu
* Cosmetic tweaks.
---------
Co-authored-by: Laurent <laurent.mazare@gmail.com >
2023-11-04 21:26:41 +01:00
dc68c130e4
Support more ONNX ops. ( #1267 )
...
* Add LogSoftmax.
* Support for Transpose.
2023-11-04 15:10:14 +01:00
bc9a1bf239
Improve the ONNX basic example + bugfixes ( #1266 )
...
* Generate some zeros tensor in the onnx simple-eval example.
* Fix the casting operation.
* Support more ops.
* Handle reshape.
* Concat.
* Softmax.
2023-11-04 10:02:47 +01:00
f7c957d64f
ONNX casting support. ( #1265 )
...
* ONNX casting support.
* Handle tensor constants.
* Bugfix the binary ops.
2023-11-04 08:34:24 +01:00
8cbb9d0e6c
Add some preliminary ONNX support ( #1260 )
...
* Add the onnx protos.
* Move the reading bits.
* Install protoc on the CI.
* Install protoc on the cuda CI too.
* Use clap for the onnx tool.
* Tweak the CI protoc install.
* Add some simple evalution function.
* Add some binary operator support.
2023-11-04 06:36:05 +01:00
bfe95115c6
Update README.md ( #1264 )
2023-11-04 05:32:32 +01:00
6fa3151820
Allow using gguf-v3 files. ( #1262 )
2023-11-03 23:07:53 +01:00
0a58886ccb
add distil-whisper link ( #1261 )
2023-11-03 21:34:42 +01:00
3173b1ce3b
feat: impl backprop for erf and gelu-erf ( #1258 )
...
* impl backprop for erf anf gelu-erf
* feat: unary tests added for erf and gelu-erf
* fix: (clippy) remove immediately dereferenced ref
* fix: improve comments with pytorch code snippet
* fix: adjust comment typo in backprop impl
2023-11-03 21:32:30 +01:00
ad63f20781
add Kalosm to the list of external resources ( #1257 )
2023-11-03 19:16:46 +01:00
1cfc5d6d0c
Backprop support for conv1d (cpu only for now). ( #1255 )
2023-11-03 14:23:53 +01:00
b07b2350b6
Test for the transposed conv1d. ( #1254 )
2023-11-03 13:10:28 +01:00
1b5063f3ca
Add vllm external resource ( #1253 )
2023-11-03 12:40:31 +01:00
3b0d1e7d03
Transposed conv1d in candle-nn. ( #1252 )
2023-11-03 11:18:25 +01:00
be4555c5a5
Add the conv-transpose1d op. ( #1251 )
...
* Skeleton structure for conv-transpose1d.
* CPU implementation for conv-transpose1d.
2023-11-03 09:44:46 +01:00
6975c65112
Share the layer-norm implementation. ( #1248 )
2023-11-03 06:30:05 +01:00
a2a20aeecc
Add the swiglu activation from the chatglm PR. ( #1246 )
2023-11-02 20:01:34 +01:00
e08fbb6543
Add support for distil whisper ( #1245 )
...
* Add support for distil-whisper.
* Add distil-large.
* Rename the large model.
2023-11-02 19:32:35 +01:00
d39d0c40fd
Add hard-sigmoid and hard-swish activations ( #1244 )
...
* Add hard-sigmoid and hard-swish activations
* Update ops.rs
* Use / rather than div.
---------
Co-authored-by: Laurent <laurent.mazare@gmail.com >
2023-11-02 18:20:27 +01:00
b97463098c
llama2-c wasm fix.
2023-11-02 10:31:47 +01:00
fbd69f952c
Lazy detach. ( #1242 )
2023-11-02 07:33:48 +00:00
6c990a33ea
Remove the unused pragma for marian. ( #1236 )
2023-11-01 20:04:52 +00:00
1704f1b3ae
Consolidate the with-tracing usage. ( #1234 )
2023-11-01 18:21:36 +00:00
693fad511c
Preliminary support for ssd1b. ( #1233 )
2023-11-01 14:37:52 +00:00
36fb84f038
Add a hack for generating random uniform/normal for f16/bf16. ( #1228 )
2023-10-31 20:27:59 +00:00
c12ad45562
Add a KV cache to marian decoding. ( #1226 )
2023-10-31 08:47:44 +00:00
7d0202710b
Instructions for generating the tokenizer configs for marian-mt. ( #1225 )
2023-10-31 07:56:26 +01:00
392a00a147
Add support for the marian base model. ( #1221 )
2023-10-30 19:20:36 +00:00
4c967b9184
Use the hub files for the marian example. ( #1220 )
...
* Use the hub files for the marian example.
* Use the secondary decoder.
* Add a readme.
* More readme.
2023-10-30 17:29:36 +00:00
c05c0a8213
PyO3: Add equal and __richcmp__ to candle.Tensor ( #1099 )
...
* add `equal` to tensor
* add `__richcmp__` support for tensors and scalars
* typo
* more typos
* Add `abs` + `candle.testing`
* remove duplicated `broadcast_shape_binary_op`
* `candle.i16` => `candle.i64`
* `tensor.nelements` -> `tensor.nelement`
* Cleanup `abs`
2023-10-30 15:17:28 +00:00
969960847a
Bugfixes for marian-mt. ( #1219 )
...
* Bugfixes for marian-mt.
* Apply the final decoding head.
* More fixes.
2023-10-30 11:44:19 +00:00
5fc66bd4ba
Support negative steps in arange. ( #1218 )
2023-10-30 07:40:54 +00:00
174b208052
PyO3: Better shape handling ( #1143 )
...
* Negative and `*args` shape handling
* Rename to `PyShapeWithHole` + validate that only one hole exists
* Regenerate stubs
---------
Co-authored-by: Laurent Mazare <laurent.mazare@gmail.com >
2023-10-29 15:41:44 +00:00
154c674a79
Add i64-abs. ( #1216 )
2023-10-29 15:28:53 +00:00
7bbde55c61
Marian MT model ( #1210 )
...
* Skeleton files for the marian MT model.
* Marian initialization.
* Implement the attention forward method.
* Forward pass for the encoder side.
* Expose the encoder and decoder.
* Start plugging the decoder.
* Forward pass for the decoder layer.
* Set up the marian example.
* Add some missing backtraces.
* Bugfix.
2023-10-29 15:12:22 +00:00
c3f2676d49
PyO3: Add CI to build & upload wheels as artifacts. ( #1215 )
...
* Add maturin ci
* fix paths
* Change sdist path
2023-10-29 13:44:05 +00:00
46d6566c99
Fix the conv2d gradient computation. ( #1214 )
2023-10-29 09:50:04 +00:00
55bc3382cf
Allow for different behavior between training and eval ( #1213 )
...
* Forward with training.
* Do not use dropout on vgg evaluation.
2023-10-29 07:53:09 +01:00
dece37c6f4
feat: implement VGG13, VGG16 and VGG19 ( #1211 )
...
* feat: implement VGG13, VGG16 and VGG19
* Cosmetic fixes.
* More cosmetic tweaks + avoid re-loading the weights on each final layer.
---------
Co-authored-by: Laurent <laurent.mazare@gmail.com >
2023-10-29 06:10:23 +00:00
498c50348c
Add DDPG and fix Gym wrapper ( #1207 )
...
* Fix Gym wrapper
- It was returning things in the wrong order
- Gym now differentiates between terminated and truncated
* Add DDPG
* Apply fixes
* Remove Result annotations
* Also remove Vec annotation
* rustfmt
* Various small improvements (avoid cloning, mutability, get clippy to pass, ...)
---------
Co-authored-by: Travis Hammond <travis.hammond@alexanderthamm.com >
Co-authored-by: Laurent <laurent.mazare@gmail.com >
2023-10-28 19:53:34 +01:00
012ae0090e
Infer the config for llama2-c. ( #1208 )
2023-10-28 19:00:39 +01:00
95a857cf57
Move the llama2-c model in transformers. ( #1205 )
2023-10-28 16:51:19 +01:00
612f5b8156
Make more models cloneable. ( #1203 )
2023-10-28 07:43:08 +01:00
ef33df7ae2
No need for the even constraint on vecdot-q40-q80. ( #1202 )
2023-10-28 07:23:59 +01:00
c8face3f95
Add the relu2 and relu6 activations. ( #1201 )
2023-10-27 20:51:16 +01:00
