llama.cpp
6e093b80 - vulkan: add Flash Attention support for BFloat16 KV cache (#23420)

Commit
105 days ago
vulkan: add Flash Attention support for BFloat16 KV cache (#23420) * vulkan: add flash attention bf16 kv support * vulkan: bf16 FA coopmat1 support * vulkan: bf16 FA coopmat2 support * fix FA bf16 f32 fallback * fix FA bf16 coopmat1 shader * fix FA bf16 coopmat2 shader * code cleanup * cleanup comment change * address feedback * add O_TYPE for cm2 FA * use O_TYPE for gqaStore function * reduce BFLOAT16 ifdefs
Author
Parents
Loading