llama.cpp
d8f30a5b - vulkan: optimize prompt processing Flash Attention for RDNA3 and RDNA4 (#31323)

Commit
2 days ago
vulkan: optimize prompt processing Flash Attention for RDNA3 and RDNA4 (#31323) * vulkan: optimize Flash Attention cm1 for RDNA3 and RDNA4 * reduce shmem use through buffer reuse * clean up and fixes * limit q_in_regs to prefill * vulkan: add flipped FA path, each subgroup owns 16 Q rows * cleanup * move flip shader into own file * replace kvsep with direct load mode * fix nvidia hang, tune AMD * only store pv in sfsh if necessary * fix skip condition * nvidia tuning * reduce occupancy on RDNA4 for large attention head sizes * fix sparse FA fp16 underflow bug
Author
Parents
Loading