llama.cpp
llama: automatically set parameters not set by the user in such a way that maximizes GPU utilization
#16653
Merged

llama: automatically set parameters not set by the user in such a way that maximizes GPU utilization #16653

JohannesGaessler
JohannesGaessler JohannesGaessler requested a review from CISC CISC 1 year ago
JohannesGaessler JohannesGaessler requested a review from slaren slaren 1 year ago
JohannesGaessler JohannesGaessler requested a review from ggerganov ggerganov 1 year ago
github-actions github-actions added ggml
JohannesGaessler JohannesGaessler changed the title llama: automatically fit parameters not set by the user to free device memory llama: automatically set parameters not set by the user in such a way that maximizes GPU utilization 1 year ago
ggerganov
ark3
ehoogeveen-medweb
JohannesGaessler
ark3
JohannesGaessler JohannesGaessler force pushed to 00fb12b3 1 year ago
JohannesGaessler
JohannesGaessler JohannesGaessler force pushed from 00fb12b3 to 172c6947 1 year ago
JohannesGaessler
ark3
JohannesGaessler
ark3
JohannesGaessler
ark3
JohannesGaessler
ark3
JohannesGaessler
ark3
JohannesGaessler JohannesGaessler force pushed from d6c2ea0a to 596a297c 1 year ago
JohannesGaessler
github-actions github-actions added examples
ark3
aviallon
JohannesGaessler JohannesGaessler force pushed from 596a297c to b88cb3e8 350 days ago
JohannesGaessler
CISC
CISC commented on 2025-12-04
ggerganov
ggerganov approved these changes on 2025-12-10
ggerganov
ggerganov commented on 2025-12-10
JohannesGaessler JohannesGaessler force pushed 343 days ago
github-actions github-actions added devops
JohannesGaessler
ggerganov
JohannesGaessler
JohannesGaessler JohannesGaessler force pushed 342 days ago
JohannesGaessler llama: automatically fit args to free memory
532be323
JohannesGaessler fix CI
8a2f3f50
JohannesGaessler hints for bug reports, ensure no reallocation
f6987d4d
JohannesGaessler fix segfault with Vulkan
c93ed495
JohannesGaessler add llama-fit-params to CI
4eac35d5
JohannesGaessler fix CI
90ba3378
JohannesGaessler fix CI
17739af3
JohannesGaessler fix CI
f4986bf7
JohannesGaessler minor adjustments
9d0a0bb4
JohannesGaessler fix assignment of 1 dense layer
97820aa7
JohannesGaessler fix logger not being reset on model load failure
c963908c
JohannesGaessler remove --n-gpu-layer hint on model load failure
7dcabc65
JohannesGaessler JohannesGaessler force pushed to 7dcabc65 341 days ago
JohannesGaessler fix llama-fit-params verbosity
9faca5a9
ggerganov
ggerganov approved these changes on 2025-12-14
JohannesGaessler fix edge case
ae534ec0
aviallon
aviallon commented on 2025-12-12
aviallon
aviallon commented on 2025-12-14
JohannesGaessler fix typo [no ci]
c1bb7c03
JohannesGaessler JohannesGaessler merged b1f3a6e5 into master 340 days ago
ggerganov
ggerganov commented on 2025-12-15
ServeurpersoCom
JohannesGaessler
rankaiyx
ServeurpersoCom
sorasoras
JohannesGaessler

Login to write a write a comment.

Login via GitHub

Assignees
No one assigned
Labels
Milestone