llama.cpp
CUDA: refactor ggml_cuda_op + lower GPU latency via quantization on main GPU and tiling
#3110

Merged

CUDA: refactor ggml_cuda_op + lower GPU latency via quantization on main GPU and tiling #3110

JohannesGaessler merged 1 commit into ggml-org:master from JohannesGaessler:cuda-tensor-tiling-4

ggerganov approved these changes on 2023-09-11

ggerganov requested a review from

slaren 2 years ago

JohannesGaessler force pushed from c42b3038 to 54f041b6 2 years ago

JohannesGaessler force pushed from 866b502e to c923de70 2 years ago

slaren commented on 2023-09-11

JohannesGaessler force pushed from c923de70 to a599006a 2 years ago

slaren approved these changes on 2023-09-11

CUDA: lower GPU latency + fix Windows performance

92687450

JohannesGaessler force pushed from a599006a to 92687450 2 years ago

JohannesGaessler merged d54a4027 into master 2 years ago

Reviewers

slaren

ggerganov

Assignees

No one assigned

Labels

None yet

Milestone

No milestone