llama.cpp
Improve cuBLAS performance by using a memory pool
#1094

Merged

Improve cuBLAS performance by using a memory pool #1094

slaren merged 4 commits into ggml-org:master from slaren:cuda-pool

Improve cuBLAS performance by using a memory pool

e8797a9a

Move cuda specific definitions to ggml-cuda.h/cu

641e9a0c

Add CXX flags to nvcc

c832e7c7

dfyz commented on 2023-04-21

ggerganov approved these changes on 2023-04-21

Change memory pool synchronization mechanism to a spin lock

d774e054

slaren merged 50cb666b into master 2 years ago

slaren deleted the cuda-pool branch 2 years ago

Reviewers

ggerganov

dfyz

Assignees

No one assigned

Labels

None yet

Milestone

No milestone