a824c5c3e3
Populate the no-speech probability.
2023-07-05 08:54:04 +01:00
d8f75ceeaa
Some polish.
2023-07-05 07:41:14 +00:00
9694e35db0
Clean the decode loop of the whisper example.
2023-07-05 08:37:26 +01:00
963c75cb89
Adding offline mode.
2023-07-05 07:19:57 +00:00
fbdabf0325
Merge pull request #73 from LaurentMazare/to-scalar-bugfix
...
Bugfix for to_scalar (use the proper start offset).
2023-07-05 06:45:34 +01:00
6d1e79d378
Bugfix for to_scalar (use the proper start offset).
2023-07-05 06:42:29 +01:00
ad0666271d
Merge pull request #72 from LaurentMazare/whisper-decode
...
More whisper decode
2023-07-05 05:53:53 +01:00
0cf096eb24
Merge pull request #71 from LaurentMazare/whisper-decode
...
Whisper decode
2023-07-05 05:52:50 +01:00
3ba4bfc501
More pretty printing.
2023-07-05 05:50:33 +01:00
8cf803d1a3
Split the model in a separate file.
2023-07-05 05:46:53 +01:00
9fe7a42895
More whisper sampling.
2023-07-04 22:18:07 +01:00
80f25e6fbb
More decoding.
2023-07-04 21:43:20 +01:00
babf3b4065
Sketch the decode step for whisper.
2023-07-04 18:25:47 +01:00
9cebf07f0d
Merge pull request #70 from LaurentMazare/whisper-mask
...
[whisper] Add the causal mask in text decoding.
2023-07-04 15:33:28 +01:00
31663bc04f
Add the causal mask in text decoding.
2023-07-04 15:25:47 +01:00
43a007cba4
Upgrading bert example to work with bert-base-uncased.
...
- Always take weights from the hub
- Optional `model_id` + `revision` to use safetensors version
potentially
- Optional loading for `bert-base-uncased` (`weight` vs `gamma`).
- Take the config from the hub.
2023-07-04 14:12:14 +00:00
04f4ef81e8
Merge pull request #68 from LaurentMazare/conv1d-stride-fix
...
Properly handle the stride in conv1d.
2023-07-04 15:07:36 +01:00
459e2e1ae3
Properly handle the stride in conv1d.
2023-07-04 15:05:04 +01:00
29a0330d6d
Merge pull request #67 from LaurentMazare/whisper
...
Sketch the whisper model.
2023-07-04 14:39:25 +01:00
c3739d001b
Get the audio-encoder to return some values.
2023-07-04 14:06:09 +01:00
b3d4d0fd0f
Very inefficient conv1d implementation.
2023-07-04 13:50:41 +01:00
99b83773b5
Small cleanup.
2023-07-04 13:21:59 +01:00
599160605c
Use the stored embeddings.
2023-07-04 13:13:28 +01:00
0d99b43792
Line up the textdecoder values with the python implementation.
2023-07-04 12:27:05 +01:00
aea090401d
Run the text decoding bit.
2023-07-04 12:03:28 +01:00
950b4af49e
Proper conv1d dispatch.
2023-07-04 11:29:28 +01:00
a424d95473
Add more of the conv1d op.
2023-07-04 11:15:45 +01:00
3aac1047fe
Sketch the conv1d op.
2023-07-04 10:52:34 +01:00
e6b01d0c18
Add the conv1d layer (but not the op).
2023-07-04 10:01:05 +01:00
d71b31144d
Add a weight extraction script.
2023-07-04 09:29:19 +01:00
c09aa4b0f4
Add more to the forward pass and fix the cross-attention.
2023-07-04 09:14:28 +01:00
998cfda9c4
Sinusoid embeddings.
2023-07-04 09:03:25 +01:00
6728a85676
Add more to the whisper inference.
2023-07-04 08:32:03 +01:00
512dd9e4d6
Flesh out the whisper example.
2023-07-04 08:01:06 +01:00
0ca2af6940
Add the residual attention block.
2023-07-04 07:43:36 +01:00
b1d42231fb
Start sketching the whisper model.
2023-07-04 07:23:02 +01:00
a8b38ff821
Merge pull request #65 from LaurentMazare/bert-batch
...
Add a batch dimension on the bert example.
2023-07-04 06:21:38 +01:00
a57b314780
Add a batch dimension on the bert example.
2023-07-04 06:10:52 +01:00
8e4d298c90
Merge pull request #64 from LaurentMazare/matmul-batch
...
Better handling of the batch dimension in matmul.
2023-07-03 22:56:13 +01:00
86d691c74c
Better handling of the batch dimension in matmul.
2023-07-03 22:51:40 +01:00
ee3f7c0269
Merge pull request #63 from LaurentMazare/minor-tweaks
...
Minor tweaks.
2023-07-03 18:41:38 +01:00
b6d179cc1c
Allow for batch dimensions in the embedding layer.
2023-07-03 18:37:40 +01:00
9784d1ed9f
Minor tweaks.
2023-07-03 18:31:55 +01:00
3905de4169
Merge pull request #61 from LaurentMazare/bert
...
Sketch some Bert model.
2023-07-03 17:14:46 +01:00
5524ca29cc
Remove the fixed length hack.
2023-07-03 17:13:23 +01:00
1ea6690557
Bugfix for transpose.
2023-07-03 17:06:23 +01:00
a7f03a7bb6
Fix the layer norm to properly handle bias.
2023-07-03 16:45:03 +01:00
f379b8feae
Get some embeddings out.
2023-07-03 16:11:16 +01:00
54850e7525
Get the tensors to be loaded properly.
2023-07-03 15:53:31 +01:00
ad52b0377c
Add the varbuilder + check shapes.
2023-07-03 15:32:20 +01:00
