llama.cpp
llama : preserve original batch order for speculative decoding layer inputs
#29019
Merged
Merged
llama : preserve original batch order for speculative decoding layer inputs #29019
ggerganov
merged
13 commits into
ggml-org:master
from
hthadicherla:fix/dflash-hidden-state-row-order
gaugarg-nv
commented
on 2026-09-17
ruixiang63
commented
on 2026-09-17
ggerganov
commented
on 2026-09-18
llama: preserve original batch order for layer inputs
77b9d118
tests: cover layer-input order across KV layouts
13d6eee8
tests: exercise layer-input ordering on CUDA devices
cdcb4c1a
llama: make layer input reordering compatible with tensor split
15703aaf
llama: restore token order for unmasked NextN embeddings
42aa4631
ggml: fix WebGPU reservation and OpenVINO hidden-state capture
f94b9af0
llama: defer regression test and backend fixes to follow-ups
6a98ce34
ruixiang63
approved these changes
on 2026-09-28
llama: keep n_embd declaration in its original position
28661bbe
ggerganov
commented
on 2026-09-30
llama : pass token count to layer input extraction
1d0d25d6
llama : name original batch indices batch_idxs
17747a2e
llama : name extracted embedding indices embd_batch_idxs
7f9d0893
llama : tag target embedding reordering
dfd2d620
llama : tag extraction and name the index capture flag
ce7c8a2f
ggerganov
approved these changes
on 2026-09-30
Reviewers
Assignees
Labels
testing
ggml
merge ready
OpenVINO
WebGPU
Milestone
No milestone
Login to write a write a comment.
Login via GitHub