cuda : fix vmm pool with multi GPU #4620
slaren
merged
13 commits into
master
from
sl/cuda-virt-pool-fixes
cuda : fix vmm pool with multi GPU
32dc09aa
hip
2c3fbf98
use recommended granularity instead of minimum
a76cadad
better error checking
6f35a4a6
fix mixtral
1659cd1b
use cudaMemcpy3DPeerAsync
865d042d
use cuda_pool_alloc in ggml_cuda_op_mul_mat
32304d79
consolidate error checking in ggml_cuda_set_device
692887fb
remove unnecessary inlines
561f1f95
style fixes
0dcc1a77
only use vmm for the main device
23c6dd67
fix scratch buffer size, re-enable vmm pool for all devices
da9fc775
ggerganov
approved these changes
on 2023-12-26
remove unnecessary check id != g_main_device
f097bed5
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub