PR #13324 vulkan: scalar flash attention implementation

vulkan: scalar flash attention implementation #13324

0cc4m merged 10 commits into ggml-org:master from jeffbolznv:scalar_fa_3

vulkan: scalar flash attention implementation

005756a2

jeffbolznv requested a review from

0cc4m 1 year ago

github-actions added Vulkan

github-actions added ggml

netrunnereve commented on 2025-05-07

vulkan: always use fp32 for scalar flash attention

3a8d954e

jeffbolznv changed the title ~~vulkan: scalar flash attention implementation~~ draft: vulkan: scalar flash attention implementation 1 year ago

vulkan: use vector loads in scalar flash attention shader

876e6617

vulkan: remove PV matrix, helps with register usage

a6c940bb

vulkan: reduce register usage in scalar FA, but perf may be slightly …

c747227a

vulkan: load each Q value once. optimize O reduction. more tuning

989bfb18

vulkan: support q4_0/q8_0 KV in scalar FA

e6609427

jeffbolznv changed the title ~~draft: vulkan: scalar flash attention implementation~~ vulkan: scalar flash attention implementation 1 year ago

CI: increase timeout to accommodate newly-supported tests

00784e3d

github-actions added devops

vulkan: for scalar FA, select between 1 and 8 rows

615958f4

vulkan: avoid using Float16 capability in scalar FA

20a6246f

0cc4m approved these changes on 2025-05-09

LostRuins approved these changes on 2025-05-09

0cc4m merged dc1d2adf into master 1 year ago

Reviewers

0cc4m

LostRuins

netrunnereve

Assignees

No one assigned

Labels

Vulkan devops ggml

Milestone

No milestone