llama.cpp
fb27a525 - TP: fix split state and granularity for fused QKV gemma4, qwen35 (#28965)

Commit
58 days ago
TP: fix split state and granularity for fused QKV gemma4, qwen35 (#28965) * model: calculate split states for attn_qkv from n_head * n_embd_head_k required for gemma4 with --fuse-qkv, where n_embd is 5376 but Q is 8192. * model: handle fused full attention layers for qwen35/qwen35moe * model: add TODO: [TAG_SPLIT_QGATE_QWEN]
Author
Parents
Loading