llama.cpp

3bce8bd5 - ggml-backend : prefetch only inputs computed by earlier splits (#32100)

ggml-backend : prefetch only inputs computed by earlier splits (#32100) * ggml-backend : prefetch only inputs computed by earlier splits The copy callback of a split runs after the prefetch and can write tensors that a later split reads, e.g. the MoE cache uploads missed experts into its bank. A prefetch of such a tensor reads it before the write. * llama : skip the MoE cache when its device cannot run the cached mul_mat_id Otherwise the scheduler runs the matmul on another backend and copies the whole bank every ubatch.