5 Commits

Author	SHA1	Message	Date
Laurent	6547c4bfc3	More kv-cache testing.	2024-09-22 12:51:07 +02:00
Laurent	9cfe3c7141	Add a RotatingKVCache.	2024-09-22 12:31:25 +02:00
Laurent Mazare	45e235a747	Simplify the KvCache api. (#2207)	2024-05-23 17:07:21 +02:00
Laurent Mazare	31cf64147b	Add a couple kv-cache helper functions. (#2206)	2024-05-23 16:21:47 +02:00
Laurent Mazare	01545f7303	Add a slice_set op. (#2193) ... * Add a slice_set op. * Add some testing. * Add the dedicated kv-cache module. * Derive debug and clone. * Expose more kv-cache functions. * Return the current data when appending. * Use the new cache in the quantized phi3 model.	2024-05-18 15:58:18 +02:00