llama.cpp
llama : accept a list of devices to use to offload a model
#10497

Merged

llama : accept a list of devices to use to offload a model #10497

slaren merged 5 commits into master from sl/llama-dev-selection

slaren added breaking change

llama : accept a list of devices to use to offload a model

f4457cb8

slaren force pushed from 8a793ce9 to f4457cb8 345 days ago

accept `--dev none` to completely disable offloading

d1dc8fbd

fix dev list with dl backends

d6cf9186

slaren force pushed from a6fc1d9f to d6cf9186 345 days ago

fix other examples

acf43cc1

github-actions added examples

github-actions added server

github-actions added ggml

ggerganov approved these changes on 2024-11-25

rename env parameter to LLAMA_ARG_DEVICE for consistency

42f61c86

slaren merged 10bce045 into master 344 days ago

slaren deleted the sl/llama-dev-selection branch 344 days ago

Reviewers

ggerganov

Assignees

No one assigned

Labels

breaking change examples server ggml

Milestone

No milestone