llama.cpp
CUDA: Prefer vector flash decoding kernel for Gemma models
#12738

Merged

CUDA: Prefer vector flash decoding kernel for Gemma models #12738

JohannesGaessler merged 2 commits into ggml-org:master from gaugarg-nv:gemma_flash_attention

Prefer vector flash decoding kernel for Gemma models

f7d07dd2

gaugarg-nv requested a review from

JohannesGaessler 1 year ago

github-actions added Nvidia GPU

github-actions added ggml

JohannesGaessler approved these changes on 2025-04-03

Update ggml/src/ggml-cuda/fattn.cu

ce71aba0

JohannesGaessler merged c262bedd into master 1 year ago

gaugarg-nv deleted the gemma_flash_attention branch 63 days ago

Reviewers

JohannesGaessler

Assignees

No one assigned

Labels

Nvidia GPU ggml

Milestone

No milestone