5 Commits

Author	SHA1	Message	Date
Laurent Mazare	13401df4d1	Add an abstract type for RmsNorm. (#499)	2023-08-18 08:52:14 +01:00
Laurent Mazare	906c0f3eb5	Remove the checkpoint conversion script. (#405) ... * Remove the checkpoint conversion script. * Remove references to the script.	2023-08-11 05:59:48 +01:00
Nicolas Patry	97181a77c0	Making multiprocess require flash-attn.	2023-07-28 23:31:24 +02:00
Nicolas Patry	ed58de7551	Fixed TP sharded version.	2023-07-27 09:58:46 +02:00
Nicolas Patry	1735e4831e	TP sharding v2	2023-07-27 09:58:14 +02:00