llama.cpp

5815ca62 - moe-cache : remove the backend logic of the CPU part from the graph

moe-cache : remove the backend logic of the CPU part from the graph The CPU part reads the host experts, so the scheduler places it on the CPU without set_tensor_backend. The cache gives no CPU map for batches that a device would offload. Graphs with a CPU part look up their own slot map, so the graph does not register the part with the cache. ggml-backend : the inputs of the next split that are already computed are copied to host memory before the current split runs, so a CPU split does not wait for the device split before it. This replaces the stash of the cache, and the copy callback is again called for host weights only.