llama.cpp

4453b535 - llama : preserve original batch order for speculative decoding layer inputs (#29019)

llama : preserve original batch order for speculative decoding layer inputs (#29019) * llama: preserve original batch order for layer inputs Assisted-by: Codex * tests: cover layer-input order across KV layouts Assisted-by: Codex * tests: exercise layer-input ordering on CUDA devices Assisted-by: Codex * llama: make layer input reordering compatible with tensor split Copy each microbatch tensor from offset zero and restore original row order after synchronization. Extend the layer-input regression to cover tensor split and repeated reads and decodes. Assisted-by: Codex * llama: restore token order for unmasked NextN embeddings Use the original-token mapping for unmasked NextN rows, including when layer-input capture is disabled. Keep masked NextN rows on the logits output mapping and preserve offset-zero tensor copies. Extend the existing regression to cover NextN alone, combined layer capture, and masked outputs with repeated decodes and getters. Validation: all 256 CPU/CUDA/tensor configurations pass. Qwen3.8-27B Q4_K_M MTP completes MT-Bench at concurrency 16 before and after. Assisted-by: Codex * ggml: fix WebGPU reservation and OpenVINO hidden-state capture Reserve WebGPU vector attention scratch across batch sizes and refresh reservations when NextN capture settings change. Preserve requested OpenVINO outputs, dynamic shapes, sequence counts, and current graph bindings. Extend existing WebGPU regression coverage and enable strict allocation checks. Assisted-by: Codex * llama: defer regression test and backend fixes to follow-ups Keep this PR focused on restoring token order for layer inputs and unmasked NextN embeddings. Remove the added regression test, OpenVINO and WebGPU changes, and the separate NextN reservation change. Assisted-by: Codex * llama: keep n_embd declaration in its original position Assisted-by: Codex * llama : pass token count to layer input extraction Assisted-by: Codex * llama : name original batch indices batch_idxs Assisted-by: Codex * llama : name extracted embedding indices embd_batch_idxs Assisted-by: Codex * llama : tag target embedding reordering Assisted-by: Codex * llama : tag extraction and name the index capture flag Assisted-by: Codex