llama.cpp
metal : add tensor API flash attention kernel for F16 KV
#29570
Merged

metal : add tensor API flash attention kernel for F16 KV #29570

ggerganov merged 4 commits into ggml-org:master from ethanhq:fa-tensor-api
ethanhq
ethanhq metal : add tensor API flash attention kernel for F16 KV
18027f9d
ethanhq ethanhq requested a review from ggerganov ggerganov 51 days ago
ethanhq ethanhq requested a review 51 days ago
github-actions github-actions added testing
github-actions github-actions added ggml
github-actions github-actions added Apple Metal
ggerganov ggerganov assigned ggerganov ggerganov 51 days ago
ggerganov
ethanhq cont : add tensor FA kernels for DK=DV=512 and DK=576, DV=512
2733cfc9
ethanhq
ggerganov
ethanhq
ggerganov
ethanhq cont : support attention sinks, ALiBi and logit softcap in the tensor…
2a83277e
ethanhq cont : add tensor FA kernel for DK=192, DV=128
7f0d4d5e
ethanhq
ggerganov
jhen0409
jhen0409
jhen0409 approved these changes on 2026-09-30
ggerganov
ggerganov approved these changes on 2026-10-02
ggerganov ggerganov merged 926862e5 into master 47 days ago
ethanhq
ggerganov
ethanhq
ggerganov ggerganov added highlight

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
Labels
Milestone