llama.cpp
b94050e8 - CUDA: use LRU based eviction for cuda graphs (#21611)

Commit
91 days ago
CUDA: use LRU based eviction for cuda graphs (#21611) * CUDA: use a ring-buffer for cuda graphs * bump limit to 128 * use LRU eviction * better naming * do periodic clean-up
Author
Parents
Loading