llama.cpp
llama: use host memory if device reports 0 memory
#18587

Open

llama: use host memory if device reports 0 memory #18587

taronaeo wants to merge 4 commits into ggml-org:master from taronaeo:fix/ggml-backend-memory

ggml: allow free = 0 and total = 0 to use host memory info

c53940ae

taronaeo requested a review from

lhez 4 days ago

taronaeo requested a review from

max-krasnyansky 4 days ago

taronaeo requested a review from

ggerganov 4 days ago

github-actions added ggml

github-actions added OpenCL

taronaeo changed the title ~~ggml: allow free = 0 and total = 0 to use host memory info~~ ggml-backend: allow free = 0 and total = 0 to use host memory info 4 days ago

ggerganov commented on 2026-01-06

ggml-backend: move memory fallback logic to llama_get_device_memory_data

84f09272

ggerganov commented on 2026-01-06

llama: fix memory logic for devices with 0 free and total

00f8eb49

ggml: revert new host memory api

05e8f735

taronaeo requested a review from

CISC 1 day ago

CISC approved these changes on 2026-01-07

taronaeo changed the title ~~ggml-backend: allow free = 0 and total = 0 to use host memory info~~ llama: use host memory if device reports 0 memory 8 hours ago

Reviewers

CISC

ggerganov

lhez

max-krasnyansky

Assignees

No one assigned

Labels

ggml OpenCL

Milestone

No milestone

llama.cpp llama: use host memory if device reports 0 memory #18587 Open

llama: use host memory if device reports 0 memory #18587

llama.cpp
llama: use host memory if device reports 0 memory
#18587

Open