llama.cpp
de6f727a
- llama: limit max outputs of `llama_context` (#23861)
Go
Login via GitHub
Home
Pricing
FAQ
Install
Login
via GitHub
Commit
View On
GitHub
Commit
92 days ago
llama: limit max outputs of `llama_context` (#23861) * llama: save more VRAM by reserving n_outputs == n_seqs when possible * add n_outputs_per_seq * move n_outputs_max to server-context * change ubatch to batch everywhere
References
#23861 - llama: limit max outputs of `llama_context`
Author
am17an
Parents
95b8b8ec
Loading