CUDA memory pool with async memory allocation/deallocation #3903
ggerganov
merged
3 commits into
ggml-org:master
from
young-developer:cuda-memory-pool
Using cuda memory pools for async alloc/dealloc.
08868a44
If cuda device doesnt support memory pool than use old implementation.
7e6f4132
young-developer
changed the title 
slaren
commented
on 2023-11-02
Removed redundant cublasSetStream
587ff3bf
slaren
approved these changes
on 2023-11-02
ggerganov
approved these changes
on 2023-11-02
Login to write a write a comment.
Login via GitHub