PR #2160 CUDA: Quantized matrix matrix multiplication

CUDA: Quantized matrix matrix multiplication #2160

JohannesGaessler merged 19 commits into ggml-org:master from JohannesGaessler:cuda-matrix-matrix-6

JohannesGaessler changed the title ~~Cuda matrix matrix 6~~ CUDA: Quantized matrix matrix multiplication 2 years ago

JohannesGaessler force pushed from 0f2a62c6 to 60df883a 2 years ago

JohannesGaessler force pushed from 60df883a to 31f229c7 2 years ago

JohannesGaessler force pushed from d8e26973 to a3b096b4 2 years ago

JohannesGaessler force pushed from 84f787eb to cf0a5051 2 years ago

JohannesGaessler force pushed from cf0a5051 to 5fa10641 2 years ago

mmq implementation for non k-quants

ddb37bf8

q6_K

4b3af63e

q2_K

5bff3df0

q3_k

a62bcc89

q4_K

b59cd1dc

vdr

5d8b3de4

q5_K

b53e7138

JohannesGaessler force pushed from 5fa10641 to b53e7138 2 years ago

faster q8_1 loading

a3505fac

loop unrolling

6808800c

add __restrict__

58daf95a

q2_K sc_high

abed4463

GGML_CUDA_MMQ_Y

3c09e11c

Updated Makefile

038ed631

Update Makefile

495c8981

DMMV_F16 -> F16

656c1ab3

Updated README, CMakeLists

aa4b2c93

JohannesGaessler marked this pull request as ready for review 2 years ago

slaren commented on 2023-07-29

Fix CMakeLists.txt

c0dfd5a5

Fix CMakeLists.txt

0b5f9891

Fix multi GPU out-of-bounds

0bb22bb4

slaren approved these changes on 2023-07-29

JohannesGaessler merged 11f3ca06 into master 2 years ago

Reviewers

slaren

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

llama.cpp CUDA: Quantized matrix matrix multiplication #2160 Merged

CUDA: Quantized matrix matrix multiplication #2160

llama.cpp
CUDA: Quantized matrix matrix multiplication
#2160

Merged