llama.cpp
e7e22302 - llama : move the single-sequence prompt reserve into the model classes

Commit
2 hours ago
llama : move the single-sequence prompt reserve into the model classes Kimi Linear and MiniMax-01 reserve the prompt graph with one sequence, because inp_diag_decay grows with n_seq_tokens^2. The virtual llama_model::reserve_pp_single_seq() replaces the switch on the architecture.
Author
Parents
Loading