llama.cpp
926862e5 - metal : add tensor API flash attention kernel for F16 KV (#29570)

Commit
47 days ago
metal : add tensor API flash attention kernel for F16 KV (#29570) * metal : add tensor API flash attention kernel for F16 KV * cont : add tensor FA kernels for DK=DV=512 and DK=576, DV=512 * cont : support attention sinks, ALiBi and logit softcap in the tensor FA kernel * cont : add tensor FA kernel for DK=192, DV=128
Author
Parents
Loading