llama.cpp
vulkan: optimize prompt processing Flash Attention for RDNA3 and RDNA4
#31323

Open

vulkan: optimize prompt processing Flash Attention for RDNA3 and RDNA4 #31323

0cc4m wants to merge 14 commits into master from 0cc4m/vulkan-flash-attn-rdna-opt

vulkan: optimize Flash Attention cm1 for RDNA3 and RDNA4

075deb79

reduce shmem use through buffer reuse

c02faf8f

clean up and fixes

aee457d0

limit q_in_regs to prefill

cdf118a7

vulkan: add flipped FA path, each subgroup owns 16 Q rows

b91f7de7

cleanup

d5219c8f

move flip shader into own file

61a57498

replace kvsep with direct load mode

320e68a0

fix nvidia hang, tune AMD

d56b44d6

only store pv in sfsh if necessary

f6e5934e

fix skip condition

295921ab

nvidia tuning

7f14294d

reduce occupancy on RDNA4 for large attention head sizes

d52056be

0cc4m requested a review 6 days ago

github-actions added Vulkan

github-actions added ggml

fix sparse FA fp16 underflow bug

9731be49

Reviewers

No reviews

Assignees

No one assigned

Labels

Vulkan ggml

Milestone

No milestone