llama.cpp
d9ee007a - model : fix qwen4exp layer inputs for DFlash drafters (#30927)

Commit
28 days ago
model : fix qwen4exp layer inputs for DFlash drafters (#30927) qwen4exp published the hc-wide residual [n_embd, hc, n_tokens] as t_layer_inp, but the drafter reads layer inputs as [n_embd, n_tokens] rows: llama_context sizes each embd_layer_inp buffer as n_embd * n_batch and the DFlash encoder input is n_extract * n_embd wide. Evaluating a DFlash draft over a qwen4exp target therefore aborts on the first request in llama_context::extract_layer_inputs. Collapse the streams with a plain mean before exposing them, as deepseek4 already does for its hyper-connection residual, and build the mean only for the layers a drafter actually requested. Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB)
Author
Parents
Loading