llama.cpp
opencl: general flash attention decode performance optimizations
#25366
Merged
Merged
opencl: general flash attention decode performance optimizations #25366
lhez
merged
12 commits into
ggml-org:master
from
qualcomm:hq/fa-decode-opt
opencl: vec flash-attention decode kernels for f16/q8_0/q4_0 KV
aa378b3e
opencl: improve non FA KQ mv kernels
66e0185d
opencl: tweaks for multiquery FA
5ecca7ab
opencl: some tweaks for FA q1 kernels
aedd3d95
opencl: FA with DK=DV=512 for gemma-4
1911f760
opencl: various fixes
41a7fa89
opencl: cleanup
201a8c70
opencl: fix FA decode crash for DK=512 (gemma-4)
ac4da5a8
opencl: run DK=512 FA decode on CPU
f5c88b99
opencl: compile MQ_GQA=8 FA kernels in a minimal program
b3d4125c
opencl: remove stray token in flash_attn_f32_f16.cl
92a3fb9e
opencl: split f16-KV FA decode finer (FD_KV_PER_SPLIT_F16)
6473da6c
lhez
approved these changes
on 2026-07-06
max-krasnyansky
approved these changes
on 2026-07-06
Login to write a write a comment.
Login via GitHub