llama.cpp
CUDA: mul_mat_vec_q tiling, refactor mul mat logic
#5434

Merged

CUDA: mul_mat_vec_q tiling, refactor mul mat logic #5434

JohannesGaessler merged 7 commits into ggml-org:master from JohannesGaessler:cuda-faster-mmvq-12

CUDA: mul_mat_vec_q tiling, refactor mul mat logic

97f8a7a2

fix AMD

2bb97fca

revert low register pressure changes

76a0128b

ggerganov approved these changes on 2024-02-11

ggerganov requested a review from

slaren 2 years ago

slaren commented on 2024-02-11

refactor fp16 logic, only consider used devices

005de593

slaren commented on 2024-02-11

refactor boolean logic

b1f6fab6

any_pascal fixup

a3a46580

Update ggml-cuda.cu

763083e5

slaren approved these changes on 2024-02-11

JohannesGaessler merged 3bdc4cd0 into master 2 years ago

Reviewers

slaren

ggerganov

Assignees

No one assigned

Labels

None yet

Milestone

No milestone