llama.cpp

ae86645c - Consistify polling frequency of llama-bench with args

Consistify polling frequency of llama-bench with args llama-bench differs from llama-server in 2 ways: 1. It attaches a persistent threadpool across cgraph invocations 2. It has a polling scheme of 50 (as opposed to 0 of args) Together, this causes a significant perf regression in the CUDA backend on Windows when the CPU backend is built with OpenMP disabled. Why is the CPU backend affecting the CUDA backend? Typically, we have a LUT on the CPU backend where we map Token_ID -> Embedding to save on VRAM budget.