llama.cpp
Vulkan k-quant mmq and ggml-backend offload functionality
#6155

Merged

Vulkan k-quant mmq and ggml-backend offload functionality #6155

0cc4m merged 12 commits into master from 0cc4m/vulkan-improvements

Fix Vulkan no kv offload incoherence

492ad4b0

Add k-quant mul mat mat shaders

cb6636e0

Merge remote-tracking branch 'origin/master' into 0cc4m/vulkan-improv…

f315402d

Rework working buffer allocation, reduces vram use noticeably

86386e2c

0cc4m requested a review from

slaren 2 years ago

slaren approved these changes on 2024-03-19

Default to all dedicated GPUs

bcdd6531

Add fallback for integrated GPUs if no dedicated GPUs are found

8ddd557d

Add debug info which device is allocating memory

24e5039f

Fix Intel dequant issue

1fceeb90

Fix Vulkan GGML_OP_GET_ROWS implementation

d00b11b0

Merge upstream changes, fix conflicts

6cb07fb0

slaren commented on 2024-03-27

Clean up merge artifacts

0cda5679

Remove Vulkan warning

b7863ab7

0cc4m merged ba0c7c70 into master 2 years ago

0cc4m deleted the 0cc4m/vulkan-improvements branch 2 years ago

Reviewers

slaren

Assignees

No one assigned

Labels

None yet

Milestone

No milestone