llama.cpp
e4e9c432 - Make graph_max_nodes vary by ubatch size (#17794)

Commit
42 days ago
Make graph_max_nodes vary by ubatch size (#17794) * Make graph_max_nodes vary by ubatch size for models where chunking might explode the graph * Update src/llama-context.h Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> * Add missing const --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Author
Parents
Loading