cuda : improve cuda pool efficiency using virtual memory #4606
slaren
merged
22 commits into
master
from
sl/cuda-virt-pool
cuda : improve cuda pool efficiency using virtual memory
0d77fbd7
JohannesGaessler
commented
on 2023-12-22
fix mixtral
eb223dcd
fix cmake build
bd78dc9a
check for vmm support, disable for hip
872408cf
ggerganov
approved these changes
on 2023-12-23
fix hip build
9452d0d5
clarify granularity
20860dae
JohannesGaessler
approved these changes
on 2023-12-23
move all caps to g_device_caps
4c0f300a
refactor error checking
545f23d0
add cuda_pool_alloc, refactor most pool allocations
110b5055
fix hip build
b7da1ba0
CUBLAS_TF32_TENSOR_OP_MATH is not a macro
d8b06c21
more hip crap
9f5ac6d2
llama : fix msvc warnings
5eb62622
ggml : fix msvc warnings
6fe9da0f
minor
d8883623
Merge remote-tracking branch 'origin/master' into sl/cuda-virt-pool
26e97b58
minor
ab6ad5e6
cuda : fallback to CPU on host buffer alloc fail
5acc9e50
Update ggml-cuda.cu
b9c5a6e7
Update ggml-cuda.cu
3081c4e7
ensure allocations are always aligned
3ad45fc3
act_size -> actual_size
532cb9b9
ggerganov
approved these changes
on 2023-12-24
SomeoneSerge
commented
on 2023-12-25
Reviewers
Assignees
No one assigned
Labels
high priority
need feedback
Milestone
No milestone
Login to write a write a comment.
Login via GitHub