llama.cpp
cb5fad4c - CUDA: refactor and optimize IQ MMVQ (#8215)

Commit

1 year ago

CUDA: refactor and optimize IQ MMVQ (#8215) * CUDA: refactor and optimize IQ MMVQ * uint -> uint32_t * __dp4a -> ggml_cuda_dp4a * remove MIN_CC_DP4A checks * change default * try CI fix

References

#8215 - CUDA: refactor and optimize IQ MMVQ

Author

JohannesGaessler

Parents

dae57a1e

Files8

ggml/src
- ggml-common.h
- ggml-cuda.cu
- ggml-cuda
  - common.cuh
  - fattn-common.cuh
  - mmvq.cu
  - vecdotq.cuh
- ggml-sycl
  - mmvq.cpp
  - vecdotq.hpp

llama.cpp cb5fad4c - CUDA: refactor and optimize IQ MMVQ (#8215)

llama.cpp
cb5fad4c - CUDA: refactor and optimize IQ MMVQ (#8215)