llama.cpp
4529c660 - kv-cache: Fix state restore fragmented cache (#17982)

Commit
3 days ago
kv-cache: Fix state restore fragmented cache (#17982) * kv-cache : fix state restore with fragmented cache (#17527) Change find_slot to allow non-contiguous allocation during state restore. Fixes 'failed to find available cells in kv cache' error when restoring state to fragmented cache. * tests : update logic * cleanup: tightened state_read_meta sig, added is_contiguous case * fix: state_read_meta arg reorder loose ends --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Author
Parents
Loading