llama.cpp
improve CUDA cpy memory bandwidth when copying transposed tensor
#16841
Merged
Merged
improve CUDA cpy memory bandwidth when copying transposed tensor #16841
JohannesGaessler
merged
20 commits into
ggml-org:master
from
bssrdf:cuda-transpose-cpy
WIP
5afac4df
added a cpy kernel specific to transposed tensor which uses smem to a…
30d46071
added BF16 support
d3bdcf84
more strict check to make sure src0 is a transpose
18818a28
JohannesGaessler
commented
on 2025-10-30
reformulated to handle more complicated transpose cases
35daa02a
merged ok
29387cee
bring back 2D transpose for higher performance
d2ec251f
allow build on windows
38096455
tranpose copy more shapes
c36b70b1
minor tweak
90fd9920
final clean up
d49232c6
restore some test cases
8dbb4c7d
Merge branch 'cuda-transpose-cpy' of github.com:bssrdf/llama.cpp into…
351cf561
keep only the kernel for true tranposed case; updated with review sug…
28e5cf6e
make CI happy
1f8e4c0e
JohannesGaessler
approved these changes
on 2025-11-04
remove headers not needed
e909afd9
reduced bank conflicts for fp16 and bf16
3b8100c7
add missing const*
51a25903
now bank conflicts free
2eb51172
use padding instead of swizzling
bc95e58d
Reviewers
Assignees
No one assigned
Labels
testing
Nvidia GPU
ggml
Milestone
No milestone
Login to write a write a comment.
Login via GitHub