llama.cpp

CUDA: int8 tensor cores for MMQ (q4_K, q5_K, q6_K)

#7860

Merged