llama.cpp
[CUDA ] Write an optimized flash_attn_stream_k_fixup kernel
#21159

Merged

[CUDA ] Write an optimized flash_attn_stream_k_fixup kernel #21159

JohannesGaessler merged 5 commits into ggml-org:master from gaugarg-nv:fa_opt

gaugarg-nv requested a review 14 days ago

github-actions added Nvidia GPU

github-actions added ggml

gaugarg-nv changed the title ~~Write an optimized flash_attn_stream_k_fixup kernel~~ [CUDA ] Write an optimized flash_attn_stream_k_fixup kernel 14 days ago

JohannesGaessler commented on 2026-03-30

gaugarg-nv force pushed from d1fd632a to 25ef2dfa 9 days ago

Write an optimized flash_attn_stream_k_fixup kernel

2ab29b93

Use the new kernel only for nblocks_stream_k_raw > 4 * ntiles_dst to …

19326bae

Address review comments

bb28013b

gaugarg-nv force pushed from 35d08300 to bb28013b 9 days ago

JohannesGaessler approved these changes on 2026-04-06

ggerganov approved these changes on 2026-04-06

am17an commented on 2026-04-06

Address review comments

f4daaf5e

am17an approved these changes on 2026-04-06

Revert variable names to original

e4b95588

JohannesGaessler approved these changes on 2026-04-06

JohannesGaessler merged 15f786e6 into master 6 days ago

Reviewers

JohannesGaessler

am17an

ggerganov

Assignees

No one assigned

Labels

Nvidia GPU ggml

Milestone

No milestone

llama.cpp [CUDA ] Write an optimized flash_attn_stream_k_fixup kernel #21159 Merged

[CUDA ] Write an optimized flash_attn_stream_k_fixup kernel #21159

llama.cpp
[CUDA ] Write an optimized flash_attn_stream_k_fixup kernel
#21159

Merged