llama.cpp
cuda: tile the lightning indexer over keys and tokens for 4 heads
#29901

Merged

cuda: tile the lightning indexer over keys and tokens for 4 heads #29901

ggerganov merged 4 commits into ggml-org:master from ServeurpersoCom:cuda-lightning-indexer-tile-4-heads

cuda: tile the lightning indexer over keys and tokens for 4 heads

0628a391

ServeurpersoCom requested a review from

ggerganov 41 days ago

ServeurpersoCom requested a review 41 days ago

ServeurpersoCom requested a review from

am17an 41 days ago

github-actions added testing

github-actions added ggml

github-actions added CUDA

am17an requested a review from

fairydreaming 40 days ago

am17an commented on 2026-10-04

cuda: multiply the lightning indexer tile in float

467b4439

cuda: widen each lightning indexer key once for all heads

9a9e2b9b

am17an approved these changes on 2026-10-04

cuda: stop the lightning indexer tile from spilling registers on ROCm

5d775e32

am17an added merge ready

ggerganov merged 1b43d311 into master 39 days ago

Reviewers

am17an

ggerganov

fairydreaming

Assignees

No one assigned

Labels

testing ggml merge ready CUDA

Milestone

No milestone