llama.cpp

15e50990 - CUDA/HIP: matrix-core (MFMA) lightning indexer for CDNA2 (gfx90a) (#29050)

CUDA/HIP: matrix-core (MFMA) lightning indexer for CDNA2 (gfx90a) (#29050) * CUDA/HIP: matrix-core (MFMA) lightning indexer for CDNA2 (gfx90a) * Addressing reviewers comments, mma.cuh and physical warp size * CUDA/HIP: single mma.cuh matrix-core lightning indexer ::CDNA:: * Support RDNA3 * CUDA/HIP: stream heads through the mma lightning indexer kernel Process 16 heads at a time with double-buffered Q staging instead of keeping all heads resident in shared memory, and drop the GGML_INDEXER_NO_MFMA env. * enable for CUDA * CUDA: Remove wmma lightning-indexer, also run head 32 through mma kernel * CUDA: Nv mma dosent transpose accumulator --------- Co-authored-by: Carl Philipp Klemm <carl@uvos.xyz> Co-authored-by: Johannes Gäßler <johannesg@5d6.de>