llama.cpp
llama: automatically set parameters not set by the user in such a way that maximizes GPU utilization
#16653

Merged

llama: automatically set parameters not set by the user in such a way that maximizes GPU utilization #16653

JohannesGaessler merged 15 commits into ggml-org:master from JohannesGaessler:llama-memory-fit-9

JohannesGaessler requested a review from

CISC 87 days ago

JohannesGaessler requested a review from

slaren 87 days ago

JohannesGaessler requested a review from

ggerganov 87 days ago

github-actions added ggml

JohannesGaessler changed the title ~~llama: automatically fit parameters not set by the user to free device memory~~ llama: automatically set parameters not set by the user in such a way that maximizes GPU utilization 87 days ago

JohannesGaessler force pushed to 00fb12b3 82 days ago

JohannesGaessler force pushed from 00fb12b3 to 172c6947 77 days ago

JohannesGaessler force pushed from d6c2ea0a to 596a297c 65 days ago

github-actions added examples

JohannesGaessler force pushed from 596a297c to b88cb3e8 40 days ago

CISC commented on 2025-12-04

ggerganov approved these changes on 2025-12-10

ggerganov commented on 2025-12-10

JohannesGaessler force pushed from 3b66926e to d309a482 33 days ago

github-actions added devops

JohannesGaessler force pushed from 00201b7c to e7c7160c 31 days ago

llama: automatically fit args to free memory

532be323

fix CI

8a2f3f50

hints for bug reports, ensure no reallocation

f6987d4d

fix segfault with Vulkan

c93ed495

add llama-fit-params to CI

4eac35d5

fix CI

90ba3378

fix CI

17739af3

fix CI

f4986bf7

minor adjustments

9d0a0bb4

fix assignment of 1 dense layer

97820aa7

fix logger not being reset on model load failure

c963908c

remove --n-gpu-layer hint on model load failure

7dcabc65

JohannesGaessler force pushed from bda5bd86 to 7dcabc65 30 days ago

fix llama-fit-params verbosity

9faca5a9

ggerganov approved these changes on 2025-12-14

fix edge case

ae534ec0

aviallon commented on 2025-12-12

aviallon commented on 2025-12-14

fix typo [no ci]

c1bb7c03

JohannesGaessler merged b1f3a6e5 into master 30 days ago

ggerganov commented on 2025-12-15

Reviewers

ggerganov

CISC

aviallon

slaren

Assignees

No one assigned

Labels

examples devops ggml

Milestone

No milestone

llama.cpp llama: automatically set parameters not set by the user in such a way that maximizes GPU utilization #16653 Merged

llama: automatically set parameters not set by the user in such a way that maximizes GPU utilization #16653

llama.cpp
llama: automatically set parameters not set by the user in such a way that maximizes GPU utilization
#16653

Merged