llama.cpp

806d397c - parallel : try smaller batches when the KV cache is fragmented