5 Commits

Author	SHA1	Message	Date
Juarez Bochi	18d30005c5	Add support to UL2 model family (#1300) ... * Add support to UL2 model family * Update docs with UL2 * Create ActivationWithOptionalGating to avoid polluting activations * Also refactor quantized t5 * Remove useless conversion * Revert Activation::NewGelu name change * Remove useless return * Apply rustfmt and clippy recommendations * Reuse t5::ActivationWithOptionalGating in quantized version * (cosmetic change) use a match rather than ifs + avoid early returns. --------- Co-authored-by: Laurent <laurent.mazare@gmail.com>	2023-11-09 18:55:09 +01:00
Juarez Bochi	d5c2a7b64b	Add info about MADLAD-400 in readme files (#1287)	2023-11-07 15:21:59 +01:00
Laurent Mazare	1a276b5da7	Add a KV cache to T5. (#873) ... * Add a KV cache to T5. * Suggest using release mode. * Use the kv cache in decoding. * Add a comment.	2023-09-17 08:00:45 +01:00
Juarez Bochi	3e49f8fce5	Implement T5 decoding (#864) ... * Load t5 decoder * Run enc, dec, and lm head, but no cross attn * Cross-attention over key_value_states * New arg for decoder input ids * Add mask, don't forward position biases through decoder * Update t5 examples * Clippy + rustfmt	2023-09-15 22:05:12 +02:00
Juarez Bochi	9daa6dbe87	Extract T5 module and add main function to use it (#829) ... * Extract t5 out of musicgen * Add main for t5 module	2023-09-13 07:14:05 +01:00